System

Abstract

A system is provided.SOLUTION: A system including means for a user to start a nurse call at a terminal, means for the terminal to transmit identification information of a patient and voice data to a server, means for the server to analyze the voice data using generative artificial intelligence and evaluate request content and urgency, means for the server to determine a priority based on the urgency, means for the server to transmit a notification according to the priority to a terminal of a nurse, and means for the nurse to report a handling status to the server through the terminal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Nurses are busy, and a major challenge is the significant time and effort they must expend responding to nurse calls from patients. In particular, when a large amount of time is spent on calls with low urgency, there is a greater risk of delays in responding to patients who actually require emergency care. Furthermore, the lack of a system for properly prioritizing nurse calls and responding quickly increases the burden on nurses and reduces patient satisfaction. [Means for solving the problem]

[0005] To solve this problem, the present invention provides the following: a system including: a means for a user (patient) to initiate a nurse call on a terminal; a means for the terminal to send the patient's identification information and voice data to a server; a means for the server to analyze the voice data using generative artificial intelligence and evaluate the request content and urgency; a means for the server to determine a priority based on the urgency; a means for the server to send a notification according to the priority to the nurse's terminal; and a means for the nurse to report the response status to the server via their terminal. This allows for appropriate determination of the priority of nurse calls and enables quick and efficient responses, thereby reducing the burden on nurses and improving patient satisfaction.

[0006] "User" refers to a patient or nurse who uses the system.

[0007] A "terminal" is an electronic device used by a patient or nurse that has functions such as initiating a nurse call, sending data, and receiving notifications.

[0008] A "server" is a computer that manages data for the entire system and performs tasks such as analyzing voice data using generative artificial intelligence, determining priorities, and sending notifications.

[0009] "Generative artificial intelligence" refers to algorithms and software that have the ability to convert voice data into text and analyze it using natural language processing.

[0010] "Voice data" refers to data that is a digital recording of a user's voice.

[0011] "Identification information" refers to information for identifying a patient or user, and specifically refers to a radio frequency identification tag, a barcode, or the like.

[0012] "Urgency" refers to an index for evaluating the urgency of a user's request or situation.

[0013] "Priority" refers to an index that indicates the order of response and importance, determined by the server based on urgency.

[0014] "Notification" refers to messages or alerts sent by the server to the nurse's device.

[0015] "Response status" refers to information indicating the progress and completion status when a nurse responds to a nurse call. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention provides a method and system for incorporating generative artificial intelligence into a nurse call system to effectively manage patient contacts. This system works with the cooperation of users (patients), terminals, a server, and nurses.

[0038] System configuration

[0039] 1. User initiates nurse call

[0040] The user initiates contact by pressing the nurse call button on the terminal.

[0041] 2. Terminal data collection and transmission

[0042] The terminal obtains the user's identification information (eg, a radio frequency identification tag or a barcode) and collects the user's voice data.

[0043] The terminal transmits the acquired identification information and voice data to the server.

[0044] 3. Server Data Analysis

[0045] The server passes the received voice data to the generative artificial intelligence.

[0046] Generative AI uses speech recognition technology to convert voice data into text, and then uses natural language processing technology to analyze the text data, understand the content, and assess its urgency.

[0047] 4. Server Prioritization

[0048] The server evaluates the urgency of the nurse call and determines its priority based on the analysis results of the generative artificial intelligence.

[0049] 5. Sending notifications

[0050] The server sends notifications to the nurses' terminals based on the determined priority: high-priority requests are notified immediately, and low-priority requests are notified at an appropriate time.

[0051] 6. Nurse response and reporting

[0052] The user (nurse) receives the notification from the terminal and responds according to priority.

[0053] Nurses report their response status from their terminals to the server, and the progress and completion status of the response are updated.

[0054] Specific examples

[0055] Example 1: Emergency nurse call

[0056] User (patient) presses nurse call button:

[0057] Patient: "My stomach hurts, can you do something about it?"

[0058] The device sends the information to the server:

[0059] Patient ID: 12345

[0060] Audio data: "My stomach hurts, can you help me?"

[0061] The server analyzes the data using generative artificial intelligence:

[0062] Generative AI: Analyzes the content of voice data and determines "pain," "abdomen," and "urgency."

[0063] Urgency: High

[0064] The server decides the priority:

[0065] Priority: High

[0066] Immediate notification to nurses

[0067] The nurse begins to respond:

[0068] A nurse immediately rushes to the patient's room and provides treatment.

[0069] Sends a timestamp of when the response started to the server

[0070] Status report after completion of response:

[0071] The nurse completes the procedure and sends the end timestamp to the server.

[0072] The server logs the completion of the process and updates the emergency response history.

[0073] Example 2: Non-emergency nurse call

[0074] User (patient) presses nurse call button:

[0075] Patient: "What time is my next appointment?"

[0076] The device sends the information to the server:

[0077] Patient ID: 67890

[0078] Audio data: "What time is my next appointment?"

[0079] The server analyzes the data using generative artificial intelligence:

[0080] Generative artificial intelligence: Analyzes the content of voice data and determines "consultation time" and "information provision."

[0081] Urgency: Low

[0082] The server decides the priority:

[0083] Priority: Low

[0084] Providing timely notification to nurses

[0085] The nurse begins to respond:

[0086] The nurse will contact the patient after other emergency procedures have been completed.

[0087] Sends a timestamp of when the response started to the server

[0088] Status report after completion of response:

[0089] The nurse completes the information provision and sends the end timestamp to the server.

[0090] The server logs the completion of the process and updates the non-emergency history.

[0091] In this way, the system of the present invention is designed to improve nurse efficiency and increase patient satisfaction.

[0092] The processing flow will be explained below.

[0093] Step 1:

[0094] The user presses the nurse call button on the terminal, which sends out a signal to start a nurse call. Specifically, the terminal is waiting to receive user input and detects that the button has been pressed.

[0095] Step 2:

[0096] The terminal acquires the patient's identification information, which is automatically read using a radio frequency identification tag or barcode reader.

[0097] Step 3:

[0098] The user communicates their inquiry or request to the terminal by voice. For example, they can input a message such as "My stomach hurts."

[0099] Step 4:

[0100] The device collects the audio data and temporarily stores it as an audio file, which is accompanied by a timestamp and patient identification information.

[0101] Step 5:

[0102] The device sends the voice data and identification information to the server using the secure HTTPS protocol, and the data is encrypted.

[0103] Step 6:

[0104] The server passes the received voice data to the generative AI, which has a dedicated module for analyzing the voice data.

[0105] Step 7:

[0106] Generative artificial intelligence uses speech recognition technology to convert audio data into text, which is then passed on to the next processing stage for analysis.

[0107] Step 8:

[0108] Generative AI analyzes text data using natural language processing (NLP) techniques to assess the patient's needs and urgency, detecting keywords such as "pain," "fall," and "dyspnea."

[0109] Step 9:

[0110] Based on the results of the generative AI analysis, the server evaluates the urgency and determines the priority of the response, which is classified as "high," "medium," or "low."

[0111] Step 10:

[0112] The server sends notifications to the nurse's terminal according to priority. High-priority requests are notified immediately, while medium- and low-priority requests are notified at an appropriate time.

[0113] Step 11:

[0114] The terminal (nurse's device) receives notifications from the server and notifies the nurse with audio and visual alerts, including the request and priority.

[0115] Step 12:

[0116] The user (nurse) checks the notification and starts responding based on the specified priority. When the nurse starts responding, they input the start of response into the terminal.

[0117] Step 13:

[0118] The device sends a timestamp of when the response started to the server, and the server confirms that the response started and records it in a log.

[0119] Step 14:

[0120] The user (nurse) visits the patient and responds. Depending on the response, necessary measures are taken.

[0121] Step 15:

[0122] When the user (nurse) completes the treatment, they input the completion of the treatment into the terminal. The terminal sends a timestamp of the completion of the treatment to the server.

[0123] Step 16:

[0124] The server receives the timestamp of the response completion and updates the response history, thereby recording the nurse call processing status as completed in the log.

[0125] This processing flow allows the nurse call system to efficiently process patient requests, reducing the burden on nurses and improving patient satisfaction.

[0126] Example 1

[0127] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0128] In conventional nurse call systems, nurses may not be able to respond appropriately to calls from patients with different levels of urgency. This makes it difficult to respond quickly to calls with high urgency, which can lead to a decrease in patient safety and satisfaction. Furthermore, nurses may not be able to perform their work efficiently, which can result in excessive burdens. Therefore, the present invention aims to provide a system that effectively manages calls from patients, reducing the workload of nurses while enabling them to respond quickly and accurately.

[0129] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0130] In this invention, the server includes a means for a user to initiate a nurse call on a terminal, a means for the terminal to send the user's identification information and voice data to the server, a means for the server to analyze the voice data using generative artificial intelligence and evaluate the request content and urgency, a means for the server to determine the priority of the nurse call based on the analysis results, a means for the server to send a notification according to the urgency to the nurse's terminal, a means for the nurse to report the response status to the server via the terminal, and a means for the server to record the completion of processing in a log and update the response history. This enables a quick and accurate response that takes the urgency into consideration and reduces the workload of nurses.

[0131] "User" refers to the person who operates the terminal that initiates the nurse call.

[0132] "Terminal" refers to a device that acquires user identification information and voice data and transmits them to a server.

[0133] "Identification information" is unique information for identifying a user, and is obtained using, for example, a radio frequency identification tag or a barcode.

[0134] "Voice data" refers to data including what the user said when calling the nurse.

[0135] "Server" refers to a device or system that receives data sent from a terminal and analyzes it using generative artificial intelligence.

[0136] "Generative artificial intelligence" refers to an artificial intelligence model that uses voice recognition technology and natural language processing technology to analyze voice data.

[0137] "Voice recognition technology" refers to technology that converts voice data into text data.

[0138] "Natural language processing technology" refers to technology for analyzing text data and understanding its content.

[0139] "Urgency" refers to an indicator that evaluates how urgent the content of a nurse call is.

[0140] "Priority" refers to the criteria for determining the order of response based on the urgency of the nurse call.

[0141] "Notification" refers to a message that sends information about the nurse call and the level of urgency to the nurse's terminal.

[0142] "Caregiver" refers to a medical professional who takes action based on the notification sent.

[0143] "Response status" refers to the progress of a nurse's response to a nurse call.

[0144] "Log" refers to data that a server uses to record the completion of processing and response history.

[0145] This invention provides a method and system for incorporating generative artificial intelligence into a nurse call system to effectively manage calls from patients. This system works through the cooperation of users (patients), terminals, a server, and caregivers (nurses).

[0146] The system is implemented using the following hardware and software.

[0147] Terminal: A device that acquires a user's identification information and voice data and transmits them to a server. Specifically, a smart device equipped with a radio frequency identification tag reader or barcode reader is used.

[0148] Server: A device that uses generative artificial intelligence to process and analyze data received from the terminal, and sends notifications to the caregiver's terminal based on the determined priority. Specific implementation examples include cloud servers and on-premise servers.

[0149] Generative artificial intelligence: Software that analyzes audio data using speech recognition technology (e.g., Google® Cloud Speech-to-Text) and natural language processing technology (e.g., Google Cloud Natural Language API).

[0150] This system allows users (patients) to quickly call a nurse in an emergency, and nurses can respond efficiently by receiving notifications according to priority.

[0151] The specific operation flow is shown below.

[0152] 1. User (patient) initiates nurse call

[0153] The user presses the nurse call button on the terminal to initiate a call. For example, a patient presses the nurse call button when they feel a stomach ache.

[0154] 2. Terminal data collection and transmission

[0155] The terminal acquires the user's identification information (e.g., radio frequency identification tag or barcode) and collects the user's voice data. For example, it reads "Patient ID: 12345" and sends the voice data "My stomach hurts. Can you do something about it?" to the server.

[0156] 3. Server Data Analysis

[0157] The server passes the received voice data to a generative AI. The generative AI converts the voice data into text using speech recognition technology, and then analyzes the text data using natural language processing technology to understand the content and assess the urgency. For example, the voice data "My stomach hurts. Can you do something?" would be analyzed as "pain," "abdomen," and "high urgency."

[0158] 4. Server Prioritization

[0159] The server evaluates the urgency of the nurse call based on the analysis results of the generative AI and determines the priority. For example, if the urgency is evaluated as "high," the priority will be set to "high."

[0160] 5. Sending notifications

[0161] The server sends a notification to the nurse's terminal based on the determined priority. For example, if the priority is "High," an immediate notification is sent. The notification content may include something like, "Please go immediately to the room of patient ID: 12345. The patient is complaining of stomach pain."

[0162] 6. Caregiver response and reporting

[0163] The user (caregiver) receives the notification from the terminal and responds according to priority. For example, the nurse who received the notification rushes to the patient's room. The caregiver reports the response status from the terminal to the server, records the completion of the process in a log, and updates the response history. For example, timestamps at the start and end of the response are sent to the server.

[0164] Specific examples

[0165] Example 1: Emergency nurse call

[0166] User (patient) presses nurse call button:

[0167] Patient: "My stomach hurts, can you do something about it?"

[0168] The device sends the information to the server:

[0169] Patient ID: 12345

[0170] Audio data: "My stomach hurts, can you help me?"

[0171] The server analyzes the data using generative artificial intelligence:

[0172] Generative AI: Analyzes the content of the voice data and determines whether the condition is "painful," "abdominal," or "high urgency."

[0173] Urgency: High

[0174] The server decides the priority:

[0175] Priority: High

[0176] Immediate notification to caregivers

[0177] Caregiver initiates response:

[0178] A caregiver immediately goes to the patient's room and provides treatment.

[0179] Sends a timestamp of when the response started to the server

[0180] Status report after completion of response:

[0181] The caregiver completes the procedure and sends the completion timestamp to the server.

[0182] The server logs the completion of the process and updates the emergency response history.

[0183] Example 2: Non-emergency nurse call

[0184] User (patient) presses nurse call button:

[0185] Patient: "What time is my next appointment?"

[0186] The device sends the information to the server:

[0187] Patient ID: 67890

[0188] Audio data: "What time is my next appointment?"

[0189] The server analyzes the data using generative artificial intelligence:

[0190] Generative artificial intelligence: Analyzes the content of voice data and determines "consultation time" and "information provision."

[0191] Urgency: Low

[0192] The server decides the priority:

[0193] Priority: Low

[0194] Providing timely notification to caregivers

[0195] Caregiver initiates response:

[0196] After other emergency procedures are completed, the caregiver contacts the patient.

[0197] Sends a timestamp of when the response started to the server

[0198] Status report after completion of response:

[0199] The caregiver completes the information submission and sends an end timestamp to the server.

[0200] The server logs the completion of the process and updates the non-emergency response history.

[0201] In this manner, the system of the present invention is designed to improve caregiver efficiency and increase patient satisfaction.

[0202] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0203] Step 1:

[0204] User (patient) initiates nurse call

[0205] This step begins when the user presses the nurse call button on the terminal. Specifically, the patient presses the button when they feel a stomach ache.

[0206] Input: User (patient) presses nurse call button

[0207] Output: Nurse call start signal

[0208] Step 2:

[0209] Terminal data collection and transmission

[0210] Upon receiving the nurse call start signal, the terminal simultaneously acquires the user's identification information (e.g., radio frequency identification tag or barcode) and collects the user's voice data. This data is then sent to the server. Specifically, the terminal reads "Patient ID: 12345" and sends the voice data "My stomach hurts. Can you do something about this?" to the server.

[0211] Input: Nurse call start signal, user identification information, voice data

[0212] Output: Sending a data packet containing the identification information and audio data to the server

[0213] Step 3:

[0214] Receiving data from the server and preparing for analysis

[0215] The server receives the data packets sent from the terminal and prepares them for analysis. In this step, preprocessing is performed to pass the received data to the generative artificial intelligence.

[0216] Input: Data packet containing identification information and voice data

[0217] Output: Data ready for analysis

[0218] Step 4:

[0219] Data analysis using generative artificial intelligence

[0220] The server passes the prepared data to the generative AI, which analyzes the voice data. The generative AI uses speech recognition technology to convert the voice data into text data, and then uses natural language processing technology to analyze the text data and evaluate the request's content and urgency. For example, it analyzes the voice data "My stomach hurts. Can you do something about this?" and determines that the request is "pain," "abdomen," and "high urgency."

[0221] Input: Data ready for analysis

[0222] Output: Text data and a severity rating based on that text

[0223] Step 5:

[0224] Server Prioritization

[0225] The server evaluates the urgency of the nurse call based on the analysis results of the generative artificial intelligence and determines the priority based on that evaluation. For example, if the urgency is determined to be "high," the priority is set to "high."

[0226] Input: Text data and urgency assessment using generative AI

[0227] Output: Priority determination

[0228] Step 6:

[0229] Sending notifications

[0230] The server then sends a notification to the nurse's terminal based on the determined priority. For example, if the priority is "High," it will immediately send a notification stating, "Please go immediately to the room of patient ID: 12345. The patient is complaining of stomach pain."

[0231] Input: Priority determination

[0232] Output: Notification to caregiver's terminal

[0233] Step 7:

[0234] Caregiver response and reporting

[0235] The user (caregiver) receives the notification from the terminal and responds according to the priority. For example, the caregiver who receives the notification rushes to the patient's room.

[0236] Input: Notification to caregiver's terminal

[0237] Output: Action to start the response

[0238] The nurse then reports the treatment status from the terminal to the server. The progress and completion status of the treatment are recorded using timestamps. Specifically, at the start and end of the treatment, the nurse presses the "Start Treatment" and "Complete Treatment" buttons to send timestamps to the server.

[0239] Input: Response start and completion timestamps

[0240] Output: Updates the corresponding log to the server

[0241] This enables the entire system to respond quickly and appropriately, taking into account the level of urgency, reducing the workload of caregivers and increasing patient satisfaction.

[0242] (Application example 1)

[0243] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0244] Existing security systems often fail to properly detect and respond to suspicious activities and emergencies, increasing the burden on security guards. In particular, systems lack mechanisms for proper prioritization and immediate notification in situations requiring rapid response based on urgency. Furthermore, even in situations where rapid backup is required in an emergency, the time lag in manual communication is a problem.

[0245] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0246] In this invention, the server includes means for acquiring audio or video data in real time and detecting suspicious activity or emergency situations using generative artificial intelligence, means for automatically making additional backup requests in the event of an emergency, and means for analyzing the acquired audio data and evaluating the request content and urgency, thereby enabling rapid detection and response to suspicious activity or emergency situations, and rapid backup requests in the event of an emergency.

[0247] A "user" is a human entity that uses a terminal to initiate a call.

[0248] A "terminal" is an electronic device that acquires voice data and identification information and transmits them to a server.

[0249] "Identification information" refers to information that identifies a particular user or patient, and includes radio frequency identification tags and bar codes.

[0250] A "server" is a computer system that processes and manages data on a network.

[0251] "Generative AI" is an AI technology that analyzes voice and text data to understand the content and assess its urgency.

[0252] "Voice data" is data containing voice information uttered by a user.

[0253] "Urgency" is a measure of the importance and urgency of a request or event to be handled.

[0254] "Priority" refers to the order in which events or requests to be addressed are ranked according to their importance.

[0255] "Notification" is the act of informing a terminal of a specific situation by sending information from a server to the terminal.

[0256] "Response status" is information that indicates the progress and results of actions taken in response to requests or events.

[0257] "Audio or video data" refers to data that includes audio and video information captured in real time.

[0258] "Suspicious activity" is any suspicious behavior, including fraudulent or unusual actions or movements.

[0259] An "emergency" is an urgent situation requiring immediate action.

[0260] A "backup request" is a request for additional assistance or response in an emergency.

[0261] The system for realizing this invention is mainly composed of three elements: a user, a terminal, and a server. A specific embodiment of this system will be described below.

[0262] The basic configuration of the system is that the user initiates a call using their device, which then sends identification information and voice data to the server. The server uses generative artificial intelligence to analyze the voice data and evaluate the request content and urgency. Based on the analysis results, the server determines a priority according to the urgency and sends a notification to the responder's device. The responder receives the notification and reports the response status to the server, improving overall work efficiency.

[0263] The server also captures audio and video data in real time and uses generative artificial intelligence to detect suspicious activity or emergencies. If suspicious activity or an emergency is detected, the server immediately assesses the severity and automatically requests additional backups if necessary.

[0264] Hardware and Software Use

[0265] 1. Device:

[0266] Wearable devices such as smartphones, smart glasses, and head-mounted displays will be used.

[0267] It is equipped with a microphone and a camera for capturing audio and video data.

[0268] 2. Software:

[0269] Speech Recognition: Use the SpeechRecognition library to convert voice data into text.

[0270] Sending data: Use the requests library to send data from the terminal to the server.

[0271] Generative AI: Using artificial intelligence models, such as GPT-3 (registered trademark), to perform advanced analysis.

[0272] Specific examples

[0273] For example, imagine a suspicious individual appears while a security guard at a shopping mall is patrolling the area wearing smart glasses. If the suspicious individual utters a clear indication of fraudulent activity, such as "I'm going to break the camera," the microphone in the smart glasses captures the audio and immediately sends it to the server. The server then uses generative artificial intelligence to analyze the audio and evaluate whether or not there is fraudulent activity and the urgency of the situation. It then determines a priority and sends an immediate notification to the security guard's device. The guard who receives this notification can respond quickly and report the response status to the server in real time, thereby improving the overall security level.

[0274] Prompt Sentence Examples

[0275] Dear customer, please tell us more about your current situation. Is it an emergency? Do you need assistance?

[0276] Based on these specific examples, this system can realize a swift and appropriate response according to the level of urgency, which will significantly improve the security of facilities and locations, enabling effective measures to be taken in situations where a rapid response is required.

[0277] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0278] Step 1:

[0279] A user initiates a call on a terminal. The input is the user pressing the call button, and the terminal starts capturing audio. The output is the captured audio data and the user's identification information.

[0280] Step 2:

[0281] The terminal sends the user's identification information and voice data to the server. The input is the voice data and identification information acquired by the terminal, which are then sent to the server via the network. The output is the voice data and identification information passed to the server.

[0282] Step 3:

[0283] The server analyzes the voice data using a generative artificial intelligence model. The voice data is sent as input, and the generative artificial intelligence is used to convert the voice into text and analyze the content. Specifically, the voice data is converted into text data using voice recognition technology, and then the content of the text is understood using natural language processing technology, and the request content and urgency are evaluated. The analysis results are generated as output.

[0284] Step 4:

[0285] Based on the analysis results, the server evaluates the request content and urgency and determines the priority. The input is the analysis results from the generative AI, and based on this, the urgency is evaluated and priorities are set. For example, an evaluation such as "high urgency" or "low urgency" is made. A priority list is generated as the output.

[0286] Step 5:

[0287] The server sends notifications to the responder's device according to the priority. The input is a priority list, and notifications are sent to the responder's device based on this information. If the priority is high, immediate notification is sent, and if the priority is low, timely notification is sent. The output is a notification that arrives on the responder's device.

[0288] Step 6:

[0289] The responder reports the response status to the server via the terminal. The input includes details of the response performed by the responder and the progress status, which are reported via the terminal. Specifically, the timestamps for when the response started and when it was completed are reported. The output is a record of the response status on the server.

[0290] Step 7:

[0291] The server acquires audio or video data in real time and uses generative artificial intelligence to detect suspicious activity or emergencies. Real-time audio or video data is input, which is then analyzed by generative artificial intelligence. Specifically, voice recognition and video analysis are performed to detect suspicious sounds and behavior. The output is a detection result of suspicious activity or an emergency.

[0292] Step 8:

[0293] In an emergency, the server automatically requests additional backups. The input is the detection result of suspicious activity or an emergency, and the backup request is made according to the level of urgency. Specifically, if the level of urgency is high, a request for assistance is automatically sent to additional responders. The output is a backup request sent to the responder.

[0294] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0295] This invention relates to a system that incorporates generative artificial intelligence and an emotion engine into a nurse call system to accurately manage patient contact and support nurses' efficient responses. This system consists of five main components: the user (patient), terminal, server, nurse, and emotion engine.

[0296] System configuration

[0297] 1. User initiates nurse call

[0298] The user initiates contact by pressing the nurse call button on the terminal. This action causes the system to start the nurse call reception process.

[0299] 2. Terminal data collection and transmission

[0300] The terminal obtains the user's identification information and collects the voice data, which is automatically read using a radio frequency identification tag or bar code reader.

[0301] The device sends the acquired identification information and voice data to the server. This data is encrypted using a secure protocol (e.g., HTTPS) and then transmitted.

[0302] 3. Server Data Analysis

[0303] The server passes the received voice data to the generative AI, which converts the voice data into text using voice recognition technology.

[0304] The server receives the converted text data, analyzes it using natural language processing (NLP) technology, and evaluates the content and urgency of the request.

[0305] 4. Emotion analysis using an emotion engine

[0306] The server passes the acquired voice and text data to the emotion engine, which analyzes the voice tone, speech content, and speech rate to recognize the user's emotions.

[0307] The server reflects the recognized emotion in the assessment of urgency and adjusts the priority as necessary.

[0308] 5. Server Prioritization

[0309] The server determines the priority of nurse call requests based on the analysis results of the generative artificial intelligence and emotion engine. Priority is classified as "high," "medium," or "low."

[0310] 6. Sending Notifications

[0311] The server sends notifications to the nurses' terminals based on the determined priority. High-priority requests are notified immediately, and medium- and low-priority requests are notified at an appropriate time.

[0312] The user (nurse) receives notifications from the device and confirms them with audio and visual alerts, which include the request, priority, and the user's emotional state.

[0313] 7. Nurse response and reporting

[0314] The user (nurse) checks the notification on the terminal and starts responding based on the priority. When the nurse starts responding, they input the start of response on the terminal.

[0315] The terminal sends a timestamp of when the response started to the server, and the response log is updated.

[0316] When the user (nurse) completes the treatment, they input the completion of the treatment into the terminal. The terminal records a timestamp of the completion of the treatment and sends it to the server.

[0317] The server receives the timestamp of the response completion and updates the response history, thereby recording the nurse call processing status as completed.

[0318] Specific examples

[0319] Example 1: Emergency nurse call

[0320] User (patient) presses nurse call button:

[0321] Patient: "My stomach hurts, can you do something about it?"

[0322] The device sends the information to the server:

[0323] Patient ID: 12345

[0324] Audio data: "My stomach hurts, can you help me?"

[0325] The server analyzes the data using generative artificial intelligence:

[0326] Generative AI: Analyzes the content of voice data and determines "pain," "abdomen," and "urgency."

[0327] Urgency: High

[0328] The server performs sentiment analysis using the emotion engine:

[0329] Emotion Engine: Analyzes voice tone and speech rate to detect the user's anxiety.

[0330] Detecting anxiety will set the level of urgency even higher.

[0331] The server decides the priority:

[0332] Priority: High

[0333] Immediate notification to nurses

[0334] The nurse begins to respond:

[0335] A nurse immediately rushes to the patient's room and provides treatment.

[0336] Sends a timestamp of when the response started to the server

[0337] Status report after completion of response:

[0338] The nurse completes the procedure and sends the end timestamp to the server.

[0339] The server logs the completion of the process and updates the emergency response history.

[0340] Example 2: Non-emergency nurse call

[0341] User (patient) presses nurse call button:

[0342] Patient: "What time is my next appointment?"

[0343] The device sends the information to the server:

[0344] Patient ID: 67890

[0345] Audio data: "What time is my next appointment?"

[0346] The server analyzes the data using generative artificial intelligence:

[0347] Generative artificial intelligence: Analyzes the content of voice data and determines "consultation time" and "information provision."

[0348] Urgency: Low

[0349] The server performs sentiment analysis using the emotion engine:

[0350] Emotion engine: Analyzes speech content and tone to determine whether the user is calm.

[0351] The server decides the priority:

[0352] Priority: Low

[0353] Providing timely notification to nurses

[0354] The nurse begins to respond:

[0355] The nurse will contact the patient after other emergency procedures have been completed.

[0356] Sends a timestamp of when the response started to the server

[0357] Status report after completion of response:

[0358] The nurse completes the information provision and sends the end timestamp to the server.

[0359] The server logs the completion of the process and updates the non-emergency history.

[0360] This system will improve the work efficiency of nurses, enable them to respond in a way that takes patients' feelings into consideration, and increase patient satisfaction.

[0361] The processing flow will be explained below.

[0362] Step 1:

[0363] The user presses the nurse call button on the terminal. This sends a signal to start the nurse call. Specifically, the terminal waits for user input and detects that the button has been pressed.

[0364] Step 2:

[0365] The terminal acquires the user's identification information. The identification information is automatically read using, for example, a radio frequency identification tag or a barcode reader. This allows identification information such as a patient ID to be acquired.

[0366] Step 3:

[0367] The user speaks their inquiry or request to the terminal. For example, they can input messages such as "I have a stomachache" or "Please tell me when my next appointment is."

[0368] Step 4:

[0369] The device collects the audio data and temporarily stores it as an audio file, which is accompanied by a timestamp and patient identification information.

[0370] Step 5:

[0371] The device sends the voice data and identification information to the server. The data is encrypted and sent securely using the HTTPS protocol.

[0372] Step 6:

[0373] The server passes the received voice data and identification information to the generative AI, which then converts the voice data into text using voice recognition technology.

[0374] Step 7:

[0375] Generative artificial intelligence uses speech recognition technology to convert voice data into text, and then passes the converted text data to the next analysis stage.

[0376] Step 8:

[0377] Generative AI analyzes text data using natural language processing (NLP) techniques to assess the patient's needs and urgency, detecting keywords such as "pain," "fall," and "dyspnea."

[0378] Step 9:

[0379] The server passes the acquired text data to the emotion engine, which analyzes the voice tone, speech content, and speech rate to recognize the user's emotions.

[0380] Step 10:

[0381] An emotion engine analyzes voice tone and speaking rate to assess the user's emotion (e.g., anxious, calm, nervous, etc.).

[0382] Step 11:

[0383] The server integrates data from the emotion engine and generative artificial intelligence, and determines priorities based on the request content, urgency, and the user's emotional evaluation. For example, if anxiety is high, the urgency level is set high.

[0384] Step 12:

[0385] The server sends notifications to the nurse's terminal based on priority: high-priority requests are notified immediately, and medium- and low-priority requests are notified at an appropriate time.

[0386] Step 13:

[0387] The terminal (nurse's device) receives notifications from the server and notifies the nurse with audio and visual alerts, including the request, priority, and the user's emotional state.

[0388] Step 14:

[0389] The user (nurse) checks the notification and starts responding based on the specified priority. When the nurse starts responding, they input the start of response into the terminal.

[0390] Step 15:

[0391] The device sends a timestamp of when the response started to the server, and the server confirms that the response started and records it in a log.

[0392] Step 16:

[0393] The user (nurse) visits the patient and provides treatment. Specifically, the user checks the patient's symptoms and provides the necessary treatment.

[0394] Step 17:

[0395] When the user (nurse) completes the treatment, they input the completion of the treatment into the terminal. The terminal records a timestamp of the completion of the treatment and sends it to the server.

[0396] Step 18:

[0397] The server receives the timestamp of the response completion and updates the response history, thereby recording the nurse call processing status as completed.

[0398] With this processing flow, the nurse call system of the present invention not only efficiently processes patient requests and reduces the burden on nurses, but also enables prompt and appropriate responses that take into consideration the patient's feelings.

[0399] Example 2

[0400] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0401] Conventional nurse call systems have difficulty accurately grasping the level of urgency and emotional state of patients, resulting in delays in nurses' responses and an inability to assign appropriate priorities. As a result, patient satisfaction has decreased and nurses' workload has increased. To solve this, a system that can respond quickly and accurately while also taking into account the patient's emotional state is needed.

[0402] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0403] In this invention, the server includes means for analyzing voice data using generative artificial intelligence and evaluating the content and urgency of requests, means for analyzing the user's emotions using an emotion analysis engine and reflecting this in the urgency evaluation, and means for determining priorities based on the urgency. This enables responses that take into account not only the content of the patient's request but also their emotional state, supporting nurses in responding quickly and appropriately, improving patient satisfaction and reducing the workload of nurses.

[0404] A "user" is a patient or a patient's representative who uses the system to make a nurse call.

[0405] A "terminal" is a device that allows a user to initiate a nurse call, and is a device that has the role of collecting patient identification information and voice data and transmitting them to a server.

[0406] The "server" is a central processing unit that receives data sent from the terminal, analyzes it using generative artificial intelligence and an emotion analysis engine, and sends appropriate notifications to nurses.

[0407] "Generative AI" is an AI technology that analyzes voice data, converts it into text, and uses natural language processing technology to evaluate the content and urgency of requests.

[0408] An "emotion analysis engine" is a technology that analyzes the tone of voice, speech content, and speech rate to recognize the user's emotional state.

[0409] "Urgency" is an evaluation index that indicates the urgency of responding to a nurse call from a user, and is an important parameter for determining the priority of nurses' responses.

[0410] "Priority" is an index that indicates the priority of the response that the server determines for a nurse call, and indicates the order in which requests are responded to.

[0411] A "nurse" is a medical professional who is responsible for responding to nurse calls from users, receiving notifications via a terminal, and taking appropriate action.

[0412] A "notification" is a message sent from the server to the nurse's terminal, with a priority set based on the degree of urgency, to prompt the nurse to take action.

[0413] This invention incorporates generative artificial intelligence and an emotion analysis engine into a nurse call system to manage patient calls with high accuracy and support nurses' efficient responses. This system consists of five main components: the user, the terminal, the server, the nurse, and the emotion analysis engine. The specific functions of each component and the operation of the entire system are explained below.

[0414] User initiates nurse call

[0415] The user initiates contact using the nurse call button. This action initiates the nurse call reception process. The user can be the patient or their representative.

[0416] Terminal data collection and transmission

[0417] When the nurse call button is pressed, the terminal collects the patient's identification information and voice data. The identification information is automatically read using a radio frequency identification tag reader or barcode reader. The terminal then transmits this information to a server. This data transmission is encrypted using a secure protocol (e.g., HTTPS).

[0418] Server data analysis

[0419] The server receives the voice data sent from the device and passes it to a generative AI. The generative AI converts the voice data into text using speech recognition technology (e.g., Google Cloud Speech-to-Text). The server then analyzes the converted text using natural language processing technology (e.g., spaCy, BERT) to evaluate the content and urgency of the request.

[0420] Emotion analysis using an emotion analysis engine

[0421] The server passes the voice and text data to an emotion analysis engine (e.g., IBM Watson (registered trademark) Tone Analyzer), which analyzes the voice tone, speech content, and speech rate to recognize the user's emotions. This allows the emotion analysis results to be reflected in the urgency assessment. If the user feels highly anxious, the urgency can be set even higher.

[0422] Prioritizing and sending notifications

[0423] The server determines the priority of the nurse call based on the analysis results and sends notifications to the nurse's device accordingly. High-priority notifications are sent immediately, while medium- and low-priority notifications are sent at an appropriate time. Notifications include the request content, priority, and the user's emotional state.

[0424] Nurse's response and report

[0425] The user (nurse) checks the notification from the terminal and begins responding based on the priority. When the nurse starts responding, they input the start of response into the terminal, and the timestamp is sent to the server. When the response ends, the end timestamp is also sent from the terminal to the server. This causes the server to update the response history and complete the entire process.

[0426] Specific examples

[0427] Example 1: Emergency nurse call

[0428] User (patient) presses nurse call button:

[0429] Patient: "My stomach hurts, can you do something about it?"

[0430] The device sends the information to the server:

[0431] Patient ID: 12345

[0432] Audio data: "My stomach hurts, can you help me?"

[0433] The server analyzes the data using generative artificial intelligence:

[0434] Generative artificial intelligence converts voice data into text and analyzes the request using natural language processing technology.

[0435] Urgency: High

[0436] The server performs sentiment analysis using the sentiment analysis engine:

[0437] The sentiment analysis engine detects the user's anxiety and further increases the level of urgency.

[0438] The server decides the priority:

[0439] Priority: High

[0440] Immediate notification to nurses

[0441] The nurse begins:

[0442] A nurse immediately rushes to the patient's room and provides treatment.

[0443] Sends a timestamp of when the response started to the server

[0444] Status report after completion of response:

[0445] The nurse completes the procedure and sends the completion timestamp to the server.

[0446] The server logs the completion of the process and updates the emergency response history.

[0447] Example 2: Non-emergency nurse call

[0448] User (patient) presses nurse call button:

[0449] Patient: "What time is my next appointment?"

[0450] The device sends the information to the server:

[0451] Patient ID: 67890

[0452] Audio data: "What time is my next appointment?"

[0453] The server analyzes the data using generative artificial intelligence:

[0454] Generative artificial intelligence converts voice data into text and analyzes the request using natural language processing technology.

[0455] Urgency: Low

[0456] The server performs sentiment analysis using the sentiment analysis engine:

[0457] The sentiment analysis engine determines that the user is calm.

[0458] The server decides the priority:

[0459] Priority: Low

[0460] Providing timely notification to nurses

[0461] The nurse begins:

[0462] The nurse will contact the patient after other emergency procedures have been completed.

[0463] Sends a timestamp of when the response started to the server

[0464] Status report after completion of response:

[0465] The nurse completes the information provision and sends the end timestamp to the server.

[0466] The server logs the completion of the process and updates the non-emergency history.

[0467] By utilizing a generative AI model and an emotion analysis engine, this system enables quick and appropriate responses that take into account the patient's emotions, thereby reducing the burden on nurses and improving patient satisfaction.

[0468] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0469] Step 1:

[0470] The user presses the nurse call button.

[0471] Specific operation: The patient initiates contact by pressing the nurse call button next to the bed, which triggers the terminal to start accepting and processing the nurse call.

[0472] Input: User action (pressing nurse call button)

[0473] Output: Nurse call start signal

[0474] Step 2:

[0475] The terminal collects patient identification information and voice data.

[0476] What it does: The device's microphone records the audio and uses a radio frequency identification tag reader or barcode reader to read the patient's identification information.

[0477] Input: Nurse call start signal

[0478] Output: Audio data and identification information

[0479] Step 3:

[0480] The terminal transmits the collected data to the server.

[0481] Specific operation: The device encrypts the collected voice data and patient identification information using the HTTPS protocol and sends it to the server.

[0482] Input: Audio data and identification information

[0483] Output: Encrypted data packet

[0484] Step 4:

[0485] The server receives the voice data and passes it to the generative artificial intelligence.

[0486] Specific operation: Voice data and identification information are stored in a database, and the voice data is sent to generative artificial intelligence for voice recognition.

[0487] Input: Encrypted data packet

[0488] Output: Text data (speech recognition results)

[0489] Step 5:

[0490] The server analyzes the text data using natural language processing technology and evaluates the request content and urgency.

[0491] What it does: The server uses a natural language processing library (e.g., spaCy) to analyze the text data and extract the necessary information (e.g., "pain," "abdomen," "urgent," etc.).

[0492] Input: Text data

[0493] Output: Request details and urgency

[0494] Step 6:

[0495] The server passes the voice and text data to an emotion analysis engine to analyze the user's emotions.

[0496] Specific operation: Analyzes voice tone, speech content, and speech rate using an emotion analysis engine (e.g., IBM Watson Tone Analyzer) to recognize the user's emotions.

[0497] Input: Audio and text data

[0498] Output: Emotional state

[0499] Step 7:

[0500] The server reflects the emotion analysis results in the urgency assessment and determines the priority.

[0501] What you do: Reassess the urgency and prioritize it as high, medium, or low based on your emotional state and the request.

[0502] Input: Request, urgency, emotional state

[0503] Output: Priority

[0504] Step 8:

[0505] The server sends a notification to the nurse's terminal.

[0506] Specific behavior: The server generates notifications based on priority and sends them to the nurse's device. High priority notifications are sent immediately.

[0507] Input: Priorities, Needs, Emotional State

[0508] Output: Notification data

[0509] Step 9:

[0510] The user (nurse) receives the notification from the terminal and begins to respond.

[0511] Specific actions: A notification appears on the nurse's device, and an audio and visual alert is sent. The nurse checks the notification and begins to respond.

[0512] Input: Notification data

[0513] Output: Now supported

[0514] Step 10:

[0515] The terminal sends a timestamp of when the response started to the server and updates the response log.

[0516] Specific operation: The nurse presses the start response button on their device, the timestamp is sent to the server, and the log is updated.

[0517] Input: Start of response

[0518] Output: timestamp

[0519] Step 11:

[0520] When the user (nurse) has completed the treatment, the completion of the treatment is input into the terminal.

[0521] Specific action: The nurse completes the procedure and presses the "procedure completed" button on the terminal.

[0522] Input: Completed

[0523] Output: Completion signal

[0524] Step 12:

[0525] The device sends a timestamp of the completion to the server and updates the response history.

[0526] Specific operation: A completion timestamp is sent from the device to the server, and the action history is recorded as completed.

[0527] Input: Completion signal

[0528] Output: Completion timestamp and updated history

[0529] (Application example 2)

[0530] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0531] Conventional factory robot maintenance systems were sometimes slow to respond appropriately when an abnormality occurred, making it difficult to respond quickly to highly urgent events. Furthermore, there was a lack of a system that could automatically evaluate the nature and urgency of the abnormality and appropriately determine priorities. This resulted in an increased burden on maintenance staff and a decline in the overall factory's production efficiency, which became an issue.

[0532] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing voice data using generative artificial intelligence and evaluating the nature of the abnormality and the level of urgency, means for determining a priority based on the level of urgency, and means for sending a notification according to the priority to a staff member's terminal. This enables a prompt and appropriate response when an abnormality occurs in a robot in a factory, reducing the burden on maintenance staff and improving factory production efficiency.

[0533] A "user" is a person or device that accesses the system and reports an abnormality, and has the role of using a terminal to transmit identification information and voice data to the server.

[0534] "Terminal" means a device operated by a user to collect and transmit identification information and voice data, including computing devices such as smartphones and tablets.

[0535] "Identification information" is information for identifying a user or a robot, and is a unique identification code obtained by a radio frequency identification tag (RFID) or a barcode.

[0536] "Voice data" refers to voice information uttered by a user or a robot, and is data that is converted into text using voice recognition technology.

[0537] A "server" is a centralized computer system that receives data sent from devices, analyzes the data using generative artificial intelligence and emotion engines, and issues instructions for appropriate responses.

[0538] "Generative AI" is a type of AI technology that analyzes input voice data, converts it into text, and uses natural language processing technology to evaluate the content of requests and abnormalities.

[0539] "Natural language processing" is a technology that enables generative artificial intelligence to analyze text data, understand its meaning, and generate appropriate responses.

[0540] An "emotion engine" is a technology that analyzes the emotions of a user or situation from voice data or text data and reflects this in the assessment of urgency.

[0541] "Priority" is an evaluation criterion that classifies the priority of a response as "high," "medium," or "low" based on the results of analysis by the server.

[0542] "Notification" is information sent to staff terminals according to the priority determined by the server, and is received as an alert or message.

[0543] "Staff" refers to the people who receive notifications from the system and take appropriate action, including maintenance workers and nurses.

[0544] The present invention relates to a robot maintenance system used in a factory that enables a quick and appropriate response when an abnormality occurs. The present invention provides a system including the following main components:

[0545] 1. User abnormality notification

[0546] The user can use the terminal to report any abnormalities in the robot. When an abnormality occurs, the user can press a button on the terminal or report the abnormality by voice, which will trigger the system to notify the abnormality.

[0547] 2. Terminal data collection and transmission

[0548] The terminal collects the user's or robot's identification information and voice data and sends it to the server. The identification information is automatically read using a radio frequency identification tag (RFID) or barcode reader. The identification information and voice data acquired by the terminal are encrypted using a secure protocol (e.g., HTTPS) and sent to the server.

[0549] 3. Server Data Analysis

[0550] The server passes the received voice data to a generative artificial intelligence (AI) system, which converts it into text using speech recognition technology. The converted text data is then analyzed using natural language processing (NLP) technology to assess the nature of the anomaly and its urgency.

[0551] 4. Emotion analysis using an emotion engine

[0552] The server passes the analyzed text data to the emotion engine, which then reflects the emotion analysis in assessing the urgency of the anomaly. The emotion engine recognizes emotions from the voice tone, speech content, speech rate, or the robot's movement patterns, and adjusts the urgency accordingly.

[0553] 5. Prioritization

[0554] The server determines the priority of anomalies based on the analysis results of the generative artificial intelligence and emotion engine, and classifies the priority as "high," "medium," or "low."

[0555] 6. Notification to staff

[0556] The server sends notifications to the maintenance staff's terminals according to the determined priority. High-priority anomalies are notified immediately, while medium- and low-priority anomalies are notified at an appropriate time. The notification includes the nature of the anomaly, priority, and emotional state.

[0557] 7. Staff Response and Reporting

[0558] The user (maintenance staff) receives the notification from the terminal and begins to respond based on priority. When the staff member starts to respond, they input "response start" into the terminal. A timestamp of the response start is sent to the server and the response log is updated. When the response is completed, they input "response completion" into the terminal and a completion timestamp is sent to the server. The server receives the response completion timestamp, updates the response history, and records the abnormality processing status.

[0559] Specific examples

[0560] As example 1, we will introduce a scenario in which an emergency robot abnormality occurs.

[0561] Robot ID: robot_123

[0562] Error message: Critical failure, motor overheating.

[0563] Analyze the details of the error message with generative AI and use an emotion engine to assess urgency and display the emotional state.

[0564] The server analyzes this error message, rates it as high urgency, and sends an immediate notification to maintenance staff.

[0565] The above configuration enables prompt and appropriate response when an abnormality occurs in a robot in a factory, thereby reducing the burden on maintenance staff and improving factory production efficiency.

[0566] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0567] Step 1:

[0568] User abnormality notification begins

[0569] When an abnormality occurs in a robot in a factory, the user can report the abnormality by pressing a dedicated button on the terminal or by voice. The input is a voice message or button operation when an abnormality occurs. The output is a signal notifying the terminal of the abnormality. This causes the system to start receiving abnormality notifications.

[0570] Step 2:

[0571] Collection of identification information and voice data

[0572] The terminal automatically reads the user's or robot's identification information using an RFID or barcode reader, and also collects voice messages from the user as voice data. The input is the RFID tag or barcode and the voice message. The output is the identification information and voice data.

[0573] Step 3:

[0574] Sending data to the server

[0575] The device sends the collected identification information and voice data to the server using a secure protocol (e.g., HTTPS). The input is the identification information and voice data. The output is the data sent to the server.

[0576] Step 4:

[0577] Analysis of audio data

[0578] The server passes the received voice data to a generative AI system, which converts the voice data into text using speech recognition technology. The input is the voice data, and the output is the converted text data. A generative AI model (e.g., GPT-4 (registered trademark)) is used for this process.

[0579] Step 5:

[0580] Text data analysis

[0581] The server analyzes the converted text data using a natural language processing (NLP) tool to evaluate the nature of the anomaly and its urgency. The input is the text data. The output is the analysis results, which include the nature of the anomaly and its urgency. An NLP tool (e.g., SpaCy) is used for this process.

[0582] Step 6:

[0583] Emotion Analysis

[0584] The server passes the analyzed text data to an emotion engine, which recognizes emotions from the voice tone and content of the speech. The input is text data. The output is the emotional state and urgency adjustment information as the result of emotion analysis. An emotion engine (e.g., IBM Watson Tone Analyzer) is used for this process.

[0585] Step 7:

[0586] Determining priorities

[0587] The server determines the priority of anomalies based on the analysis results of the generative artificial intelligence and emotion engine. The inputs are the details of the anomaly, the urgency, and the emotion analysis results. The output is the determined priority. Priorities are classified as "high," "medium," or "low."

[0588] Step 8:

[0589] Sending notifications to staff

[0590] The server sends a notification to the maintenance staff's terminal based on the determined priority. The input is the determined priority, the abnormality details, and the staff's emotional state. The output is a notification to the staff's terminal. The notification includes the abnormality details, priority, and emotional state.

[0591] Step 9:

[0592] Staff begins to respond

[0593] The user (maintenance staff) receives the notification and starts a response based on the priority. When the response starts, a timestamp of the response start is sent from the terminal to the server. The input is the notification and the start time, and the output is a response start log.

[0594] Step 10:

[0595] Response completion report

[0596] When the user (maintenance staff) completes the response, they input the response completion into the terminal and send a completion timestamp to the server. The input is the response completion time, and the output is a response completion log. The server receives the response completion timestamp, updates the response history, and records the abnormality processing status.

[0597] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0598] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0599] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0600] [Second embodiment]

[0601] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0602] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0603] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0604] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0605] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0606] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0607] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0608] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0609] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0610] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0611] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0612] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0613] This invention provides a method and system for incorporating generative artificial intelligence into a nurse call system to effectively manage patient contacts. This system works with the cooperation of users (patients), terminals, a server, and nurses.

[0614] System configuration

[0615] 1. User initiates nurse call

[0616] The user initiates contact by pressing the nurse call button on the terminal.

[0617] 2. Terminal data collection and transmission

[0618] The terminal obtains the user's identification information (eg, a radio frequency identification tag or a barcode) and collects the user's voice data.

[0619] The terminal transmits the acquired identification information and voice data to the server.

[0620] 3. Server Data Analysis

[0621] The server passes the received voice data to the generative artificial intelligence.

[0622] Generative AI uses speech recognition technology to convert voice data into text, and then uses natural language processing technology to analyze the text data, understand the content, and assess its urgency.

[0623] 4. Server Prioritization

[0624] The server evaluates the urgency of the nurse call and determines its priority based on the analysis results of the generative artificial intelligence.

[0625] 5. Sending notifications

[0626] The server sends notifications to the nurses' terminals based on the determined priority: high-priority requests are notified immediately, and low-priority requests are notified at an appropriate time.

[0627] 6. Nurse response and reporting

[0628] The user (nurse) receives the notification from the terminal and responds according to priority.

[0629] Nurses report their response status from their terminals to the server, and the progress and completion status of the response are updated.

[0630] Specific examples

[0631] Example 1: Emergency nurse call

[0632] User (patient) presses nurse call button:

[0633] Patient: "My stomach hurts, can you do something about it?"

[0634] The device sends the information to the server:

[0635] Patient ID: 12345

[0636] Audio data: "My stomach hurts, can you help me?"

[0637] The server analyzes the data using generative artificial intelligence:

[0638] Generative AI: Analyzes the content of voice data and determines "pain," "abdomen," and "urgency."

[0639] Urgency: High

[0640] The server decides the priority:

[0641] Priority: High

[0642] Immediate notification to nurses

[0643] The nurse begins to respond:

[0644] A nurse immediately rushes to the patient's room and provides treatment.

[0645] Sends a timestamp of when the response started to the server

[0646] Status report after completion of response:

[0647] The nurse completes the procedure and sends the end timestamp to the server.

[0648] The server logs the completion of the process and updates the emergency response history.

[0649] Example 2: Non-emergency nurse call

[0650] User (patient) presses nurse call button:

[0651] Patient: "What time is my next appointment?"

[0652] The device sends the information to the server:

[0653] Patient ID: 67890

[0654] Audio data: "What time is my next appointment?"

[0655] The server analyzes the data using generative artificial intelligence:

[0656] Generative artificial intelligence: Analyzes the content of voice data and determines "consultation time" and "information provision."

[0657] Urgency: Low

[0658] The server decides the priority:

[0659] Priority: Low

[0660] Providing timely notification to nurses

[0661] The nurse begins to respond:

[0662] The nurse will contact the patient after other emergency procedures have been completed.

[0663] Sends a timestamp of when the response started to the server

[0664] Status report after completion of response:

[0665] The nurse completes the information provision and sends the end timestamp to the server.

[0666] The server logs the completion of the process and updates the non-emergency history.

[0667] In this way, the system of the present invention is designed to improve nurse efficiency and increase patient satisfaction.

[0668] The processing flow will be explained below.

[0669] Step 1:

[0670] The user presses the nurse call button on the terminal, which sends out a signal to start a nurse call. Specifically, the terminal is waiting to receive user input and detects that the button has been pressed.

[0671] Step 2:

[0672] The terminal acquires the patient's identification information, which is automatically read using a radio frequency identification tag or barcode reader.

[0673] Step 3:

[0674] The user communicates their inquiry or request to the terminal by voice. For example, they can input a message such as "My stomach hurts."

[0675] Step 4:

[0676] The device collects the audio data and temporarily stores it as an audio file, which is accompanied by a timestamp and patient identification information.

[0677] Step 5:

[0678] The device sends the voice data and identification information to the server using the secure HTTPS protocol, and the data is encrypted.

[0679] Step 6:

[0680] The server passes the received voice data to the generative AI, which has a dedicated module for analyzing the voice data.

[0681] Step 7:

[0682] Generative artificial intelligence uses speech recognition technology to convert audio data into text, which is then passed on to the next processing stage for analysis.

[0683] Step 8:

[0684] Generative AI analyzes text data using natural language processing (NLP) techniques to assess the patient's needs and urgency, detecting keywords such as "pain," "fall," and "dyspnea."

[0685] Step 9:

[0686] Based on the results of the generative AI analysis, the server evaluates the urgency and determines the priority of the response, which is classified as "high," "medium," or "low."

[0687] Step 10:

[0688] The server sends notifications to the nurse's terminal according to priority. High-priority requests are notified immediately, while medium- and low-priority requests are notified at an appropriate time.

[0689] Step 11:

[0690] The terminal (nurse's device) receives notifications from the server and notifies the nurse with audio and visual alerts, including the request and priority.

[0691] Step 12:

[0692] The user (nurse) checks the notification and starts responding based on the specified priority. When the nurse starts responding, they input the start of response into the terminal.

[0693] Step 13:

[0694] The device sends a timestamp of when the response started to the server, and the server confirms that the response started and records it in a log.

[0695] Step 14:

[0696] The user (nurse) visits the patient and responds. Depending on the response, necessary measures are taken.

[0697] Step 15:

[0698] When the user (nurse) completes the treatment, they input the completion of the treatment into the terminal. The terminal sends a timestamp of the completion of the treatment to the server.

[0699] Step 16:

[0700] The server receives the timestamp of the response completion and updates the response history, thereby recording the nurse call processing status as completed in the log.

[0701] This processing flow allows the nurse call system to efficiently process patient requests, reducing the burden on nurses and improving patient satisfaction.

[0702] Example 1

[0703] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0704] In conventional nurse call systems, nurses may not be able to respond appropriately to calls from patients with different levels of urgency. This makes it difficult to respond quickly to calls with high urgency, which can lead to a decrease in patient safety and satisfaction. Furthermore, nurses may not be able to perform their work efficiently, which can result in excessive burdens. Therefore, the present invention aims to provide a system that effectively manages calls from patients, reducing the workload of nurses while enabling them to respond quickly and accurately.

[0705] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0706] In this invention, the server includes a means for a user to initiate a nurse call on a terminal, a means for the terminal to send the user's identification information and voice data to the server, a means for the server to analyze the voice data using generative artificial intelligence and evaluate the request content and urgency, a means for the server to determine the priority of the nurse call based on the analysis results, a means for the server to send a notification according to the urgency to the nurse's terminal, a means for the nurse to report the response status to the server via the terminal, and a means for the server to record the completion of processing in a log and update the response history. This enables a quick and accurate response that takes the urgency into consideration and reduces the workload of nurses.

[0707] "User" refers to the person who operates the terminal that initiates the nurse call.

[0708] "Terminal" refers to a device that acquires user identification information and voice data and transmits them to a server.

[0709] "Identification information" is unique information for identifying a user, and is obtained using, for example, a radio frequency identification tag or a barcode.

[0710] "Voice data" refers to data including what the user said when calling the nurse.

[0711] "Server" refers to a device or system that receives data sent from a terminal and analyzes it using generative artificial intelligence.

[0712] "Generative artificial intelligence" refers to an artificial intelligence model that uses voice recognition technology and natural language processing technology to analyze voice data.

[0713] "Voice recognition technology" refers to technology that converts voice data into text data.

[0714] "Natural language processing technology" refers to technology for analyzing text data and understanding its content.

[0715] "Urgency" refers to an indicator that evaluates how urgent the content of a nurse call is.

[0716] "Priority" refers to the criteria for determining the order of response based on the urgency of the nurse call.

[0717] "Notification" refers to a message that sends information about the nurse call and the level of urgency to the nurse's terminal.

[0718] "Caregiver" refers to a medical professional who takes action based on the notification sent.

[0719] "Response status" refers to the progress of a nurse's response to a nurse call.

[0720] "Log" refers to data that a server uses to record the completion of processing and response history.

[0721] This invention provides a method and system for incorporating generative artificial intelligence into a nurse call system to effectively manage calls from patients. This system works through the cooperation of users (patients), terminals, a server, and caregivers (nurses).

[0722] The system is implemented using the following hardware and software.

[0723] Terminal: A device that acquires a user's identification information and voice data and transmits them to a server. Specifically, a smart device equipped with a radio frequency identification tag reader or barcode reader is used.

[0724] Server: A device that uses generative artificial intelligence to process and analyze data received from the terminal, and sends notifications to the caregiver's terminal based on the determined priority. Specific implementation examples include cloud servers and on-premise servers.

[0725] Generative artificial intelligence: Software that analyzes audio data using speech recognition technologies (e.g., Google Cloud Speech-to-Text) and natural language processing technologies (e.g., Google Cloud Natural Language API).

[0726] This system allows users (patients) to quickly call a nurse in an emergency, and nurses can respond efficiently by receiving notifications according to priority.

[0727] The specific operation flow is shown below.

[0728] 1. User (patient) initiates nurse call

[0729] The user presses the nurse call button on the terminal to initiate a call. For example, a patient presses the nurse call button when they feel a stomach ache.

[0730] 2. Terminal data collection and transmission

[0731] The terminal acquires the user's identification information (e.g., radio frequency identification tag or barcode) and collects the user's voice data. For example, it reads "Patient ID: 12345" and sends the voice data "My stomach hurts. Can you do something about it?" to the server.

[0732] 3. Server Data Analysis

[0733] The server passes the received voice data to a generative AI. The generative AI converts the voice data into text using speech recognition technology, and then analyzes the text data using natural language processing technology to understand the content and assess the urgency. For example, the voice data "My stomach hurts. Can you do something?" would be analyzed as "pain," "abdomen," and "high urgency."

[0734] 4. Server Prioritization

[0735] The server evaluates the urgency of the nurse call based on the analysis results of the generative AI and determines the priority. For example, if the urgency is evaluated as "high," the priority will be set to "high."

[0736] 5. Sending notifications

[0737] The server sends a notification to the nurse's terminal based on the determined priority. For example, if the priority is "High," an immediate notification is sent. The notification content may include something like, "Please go immediately to the room of patient ID: 12345. The patient is complaining of stomach pain."

[0738] 6. Caregiver response and reporting

[0739] The user (caregiver) receives the notification from the terminal and responds according to priority. For example, the nurse who received the notification rushes to the patient's room. The caregiver reports the response status from the terminal to the server, records the completion of the process in a log, and updates the response history. For example, timestamps at the start and end of the response are sent to the server.

[0740] Specific examples

[0741] Example 1: Emergency nurse call

[0742] User (patient) presses nurse call button:

[0743] Patient: "My stomach hurts, can you do something about it?"

[0744] The device sends the information to the server:

[0745] Patient ID: 12345

[0746] Audio data: "My stomach hurts, can you help me?"

[0747] The server analyzes the data using generative artificial intelligence:

[0748] Generative AI: Analyzes the content of the voice data and determines whether the condition is "painful," "abdominal," or "high urgency."

[0749] Urgency: High

[0750] The server decides the priority:

[0751] Priority: High

[0752] Immediate notification to caregivers

[0753] Caregiver initiates response:

[0754] A caregiver immediately goes to the patient's room and provides treatment.

[0755] Sends a timestamp of when the response started to the server

[0756] Status report after completion of response:

[0757] The caregiver completes the procedure and sends the completion timestamp to the server.

[0758] The server logs the completion of the process and updates the emergency response history.

[0759] Example 2: Non-emergency nurse call

[0760] User (patient) presses nurse call button:

[0761] Patient: "What time is my next appointment?"

[0762] The device sends the information to the server:

[0763] Patient ID: 67890

[0764] Audio data: "What time is my next appointment?"

[0765] The server analyzes the data using generative artificial intelligence:

[0766] Generative artificial intelligence: Analyzes the content of voice data and determines "consultation time" and "information provision."

[0767] Urgency: Low

[0768] The server decides the priority:

[0769] Priority: Low

[0770] Providing timely notification to caregivers

[0771] Caregiver initiates response:

[0772] After other emergency procedures are completed, the caregiver contacts the patient.

[0773] Sends a timestamp of when the response started to the server

[0774] Status report after completion of response:

[0775] The caregiver completes the information submission and sends an end timestamp to the server.

[0776] The server logs the completion of the process and updates the non-emergency response history.

[0777] In this manner, the system of the present invention is designed to improve caregiver efficiency and increase patient satisfaction.

[0778] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0779] Step 1:

[0780] User (patient) initiates nurse call

[0781] This step begins when the user presses the nurse call button on the terminal. Specifically, the patient presses the button when they feel a stomach ache.

[0782] Input: User (patient) presses nurse call button

[0783] Output: Nurse call start signal

[0784] Step 2:

[0785] Terminal data collection and transmission

[0786] Upon receiving the nurse call start signal, the terminal simultaneously acquires the user's identification information (e.g., radio frequency identification tag or barcode) and collects the user's voice data. This data is then sent to the server. Specifically, the terminal reads "Patient ID: 12345" and sends the voice data "My stomach hurts. Can you do something about this?" to the server.

[0787] Input: Nurse call start signal, user identification information, voice data

[0788] Output: Sending a data packet containing the identification information and audio data to the server

[0789] Step 3:

[0790] Receiving data from the server and preparing for analysis

[0791] The server receives the data packets sent from the terminal and prepares them for analysis. In this step, preprocessing is performed to pass the received data to the generative artificial intelligence.

[0792] Input: Data packet containing identification information and voice data

[0793] Output: Data ready for analysis

[0794] Step 4:

[0795] Data analysis using generative artificial intelligence

[0796] The server passes the prepared data to the generative AI, which analyzes the voice data. The generative AI uses speech recognition technology to convert the voice data into text data, and then uses natural language processing technology to analyze the text data and evaluate the request's content and urgency. For example, it analyzes the voice data "My stomach hurts. Can you do something about this?" and determines that the request is "pain," "abdomen," and "high urgency."

[0797] Input: Data ready for analysis

[0798] Output: Text data and a severity rating based on that text

[0799] Step 5:

[0800] Server Prioritization

[0801] The server evaluates the urgency of the nurse call based on the analysis results of the generative artificial intelligence and determines the priority based on that evaluation. For example, if the urgency is determined to be "high," the priority is set to "high."

[0802] Input: Text data and urgency assessment using generative AI

[0803] Output: Priority determination

[0804] Step 6:

[0805] Sending notifications

[0806] The server then sends a notification to the nurse's terminal based on the determined priority. For example, if the priority is "High," it will immediately send a notification stating, "Please go immediately to the room of patient ID: 12345. The patient is complaining of stomach pain."

[0807] Input: Priority determination

[0808] Output: Notification to caregiver's terminal

[0809] Step 7:

[0810] Caregiver response and reporting

[0811] The user (caregiver) receives the notification from the terminal and responds according to the priority. For example, the caregiver who receives the notification rushes to the patient's room.

[0812] Input: Notification to caregiver's terminal

[0813] Output: Action to start the response

[0814] The nurse then reports the treatment status from the terminal to the server. The progress and completion status of the treatment are recorded using timestamps. Specifically, at the start and end of the treatment, the nurse presses the "Start Treatment" and "Complete Treatment" buttons to send timestamps to the server.

[0815] Input: Response start and completion timestamps

[0816] Output: Updates the corresponding log to the server

[0817] This enables the entire system to respond quickly and appropriately, taking into account the level of urgency, reducing the workload of caregivers and increasing patient satisfaction.

[0818] (Application example 1)

[0819] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0820] Existing security systems often fail to properly detect and respond to suspicious activities and emergencies, increasing the burden on security guards. In particular, systems lack mechanisms for proper prioritization and immediate notification in situations requiring rapid response based on urgency. Furthermore, even in situations where rapid backup is required in an emergency, the time lag in manual communication is a problem.

[0821] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0822] In this invention, the server includes means for acquiring audio or video data in real time and detecting suspicious activity or emergency situations using generative artificial intelligence, means for automatically making additional backup requests in the event of an emergency, and means for analyzing the acquired audio data and evaluating the request content and urgency, thereby enabling rapid detection and response to suspicious activity or emergency situations, and rapid backup requests in the event of an emergency.

[0823] A "user" is a human entity that uses a terminal to initiate a call.

[0824] A "terminal" is an electronic device that acquires voice data and identification information and transmits them to a server.

[0825] "Identification information" refers to information that identifies a particular user or patient, and includes radio frequency identification tags and bar codes.

[0826] A "server" is a computer system that processes and manages data on a network.

[0827] "Generative AI" is an AI technology that analyzes voice and text data to understand the content and assess its urgency.

[0828] "Voice data" is data containing voice information uttered by a user.

[0829] "Urgency" is a measure of the importance and urgency of a request or event to be handled.

[0830] "Priority" refers to the order in which events or requests to be addressed are ranked according to their importance.

[0831] "Notification" is the act of informing a terminal of a specific situation by sending information from a server to the terminal.

[0832] "Response status" is information that indicates the progress and results of actions taken in response to requests or events.

[0833] "Audio or video data" refers to data that includes audio and video information captured in real time.

[0834] "Suspicious activity" is any suspicious behavior, including fraudulent or unusual actions or movements.

[0835] An "emergency" is an urgent situation requiring immediate action.

[0836] A "backup request" is a request for additional assistance or response in an emergency.

[0837] The system for realizing this invention is mainly composed of three elements: a user, a terminal, and a server. A specific embodiment of this system will be described below.

[0838] The basic configuration of the system is that the user initiates a call using their device, which then sends identification information and voice data to the server. The server uses generative artificial intelligence to analyze the voice data and evaluate the request content and urgency. Based on the analysis results, the server determines a priority according to the urgency and sends a notification to the responder's device. The responder receives the notification and reports the response status to the server, improving overall work efficiency.

[0839] The server also captures audio and video data in real time and uses generative artificial intelligence to detect suspicious activity or emergencies. If suspicious activity or an emergency is detected, the server immediately assesses the severity and automatically requests additional backups if necessary.

[0840] Hardware and Software Use

[0841] 1. Device:

[0842] Wearable devices such as smartphones, smart glasses, and head-mounted displays will be used.

[0843] It is equipped with a microphone and a camera for capturing audio and video data.

[0844] 2. Software:

[0845] Speech Recognition: Use the SpeechRecognition library to convert voice data into text.

[0846] Sending data: Use the requests library to send data from the terminal to the server.

[0847] Generative AI: Using artificial intelligence models, such as GPT-3, to perform advanced analysis.

[0848] Specific examples

[0849] For example, imagine a suspicious individual appears while a security guard at a shopping mall is patrolling the area wearing smart glasses. If the suspicious individual utters a clear indication of fraudulent activity, such as "I'm going to break the camera," the microphone in the smart glasses captures the audio and immediately sends it to the server. The server then uses generative artificial intelligence to analyze the audio and evaluate whether or not there is fraudulent activity and the urgency of the situation. It then determines a priority and sends an immediate notification to the security guard's device. The guard who receives this notification can respond quickly and report the response status to the server in real time, thereby improving the overall security level.

[0850] Prompt Sentence Examples

[0851] Dear customer, please tell us more about your current situation. Is it an emergency? Do you need assistance?

[0852] Based on these specific examples, this system can realize a swift and appropriate response according to the level of urgency, which will significantly improve the security of facilities and locations, enabling effective measures to be taken in situations where a rapid response is required.

[0853] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0854] Step 1:

[0855] A user initiates a call on a terminal. The input is the user pressing the call button, and the terminal starts capturing audio. The output is the captured audio data and the user's identification information.

[0856] Step 2:

[0857] The terminal sends the user's identification information and voice data to the server. The input is the voice data and identification information acquired by the terminal, which are then sent to the server via the network. The output is the voice data and identification information passed to the server.

[0858] Step 3:

[0859] The server analyzes the voice data using a generative artificial intelligence model. The voice data is sent as input, and the generative artificial intelligence is used to convert the voice into text and analyze the content. Specifically, the voice data is converted into text data using voice recognition technology, and then the content of the text is understood using natural language processing technology, and the request content and urgency are evaluated. The analysis results are generated as output.

[0860] Step 4:

[0861] Based on the analysis results, the server evaluates the request content and urgency and determines the priority. The input is the analysis results from the generative AI, and based on this, the urgency is evaluated and priorities are set. For example, an evaluation such as "high urgency" or "low urgency" is made. A priority list is generated as the output.

[0862] Step 5:

[0863] The server sends notifications to the responder's device according to the priority. The input is a priority list, and notifications are sent to the responder's device based on this information. If the priority is high, immediate notification is sent, and if the priority is low, timely notification is sent. The output is a notification that arrives on the responder's device.

[0864] Step 6:

[0865] The responder reports the response status to the server via the terminal. The input includes details of the response performed by the responder and the progress status, which are reported via the terminal. Specifically, the timestamps for when the response started and when it was completed are reported. The output is a record of the response status on the server.

[0866] Step 7:

[0867] The server acquires audio or video data in real time and uses generative artificial intelligence to detect suspicious activity or emergencies. Real-time audio or video data is input, which is then analyzed by generative artificial intelligence. Specifically, voice recognition and video analysis are performed to detect suspicious sounds and behavior. The output is a detection result of suspicious activity or an emergency.

[0868] Step 8:

[0869] In an emergency, the server automatically requests additional backups. The input is the detection result of suspicious activity or an emergency, and the backup request is made according to the level of urgency. Specifically, if the level of urgency is high, a request for assistance is automatically sent to additional responders. The output is a backup request sent to the responder.

[0870] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0871] This invention relates to a system that incorporates generative artificial intelligence and an emotion engine into a nurse call system to accurately manage patient contact and support nurses' efficient responses. This system consists of five main components: the user (patient), terminal, server, nurse, and emotion engine.

[0872] System configuration

[0873] 1. User initiates nurse call

[0874] The user initiates contact by pressing the nurse call button on the terminal. This action causes the system to start the nurse call reception process.

[0875] 2. Terminal data collection and transmission

[0876] The terminal obtains the user's identification information and collects the voice data, which is automatically read using a radio frequency identification tag or bar code reader.

[0877] The device sends the acquired identification information and voice data to the server. This data is encrypted using a secure protocol (e.g., HTTPS) and then transmitted.

[0878] 3. Server Data Analysis

[0879] The server passes the received voice data to the generative AI, which converts the voice data into text using voice recognition technology.

[0880] The server receives the converted text data, analyzes it using natural language processing (NLP) technology, and evaluates the content and urgency of the request.

[0881] 4. Emotion analysis using an emotion engine

[0882] The server passes the acquired voice and text data to the emotion engine, which analyzes the voice tone, speech content, and speech rate to recognize the user's emotions.

[0883] The server reflects the recognized emotion in the assessment of urgency and adjusts the priority as necessary.

[0884] 5. Server Prioritization

[0885] The server determines the priority of nurse call requests based on the analysis results of the generative artificial intelligence and emotion engine. Priority is classified as "high," "medium," or "low."

[0886] 6. Sending Notifications

[0887] The server sends notifications to the nurses' terminals based on the determined priority. High-priority requests are notified immediately, and medium- and low-priority requests are notified at an appropriate time.

[0888] The user (nurse) receives notifications from the device and confirms them with audio and visual alerts, which include the request, priority, and the user's emotional state.

[0889] 7. Nurse response and reporting

[0890] The user (nurse) checks the notification on the terminal and starts responding based on the priority. When the nurse starts responding, they input the start of response on the terminal.

[0891] The terminal sends a timestamp of when the response started to the server, and the response log is updated.

[0892] When the user (nurse) completes the treatment, they input the completion of the treatment into the terminal. The terminal records a timestamp of the completion of the treatment and sends it to the server.

[0893] The server receives the timestamp of the response completion and updates the response history, thereby recording the nurse call processing status as completed.

[0894] Specific examples

[0895] Example 1: Emergency nurse call

[0896] User (patient) presses nurse call button:

[0897] Patient: "My stomach hurts, can you do something about it?"

[0898] The device sends the information to the server:

[0899] Patient ID: 12345

[0900] Audio data: "My stomach hurts, can you help me?"

[0901] The server analyzes the data using generative artificial intelligence:

[0902] Generative AI: Analyzes the content of voice data and determines "pain," "abdomen," and "urgency."

[0903] Urgency: High

[0904] The server performs sentiment analysis using the emotion engine:

[0905] Emotion Engine: Analyzes voice tone and speech rate to detect the user's anxiety.

[0906] Detecting anxiety will set the level of urgency even higher.

[0907] The server decides the priority:

[0908] Priority: High

[0909] Immediate notification to nurses

[0910] The nurse begins to respond:

[0911] A nurse immediately rushes to the patient's room and provides treatment.

[0912] Sends a timestamp of when the response started to the server

[0913] Status report after completion of response:

[0914] The nurse completes the procedure and sends the end timestamp to the server.

[0915] The server logs the completion of the process and updates the emergency response history.

[0916] Example 2: Non-emergency nurse call

[0917] User (patient) presses nurse call button:

[0918] Patient: "What time is my next appointment?"

[0919] The device sends the information to the server:

[0920] Patient ID: 67890

[0921] Audio data: "What time is my next appointment?"

[0922] The server analyzes the data using generative artificial intelligence:

[0923] Generative artificial intelligence: Analyzes the content of voice data and determines "consultation time" and "information provision."

[0924] Urgency: Low

[0925] The server performs sentiment analysis using the emotion engine:

[0926] Emotion engine: Analyzes speech content and tone to determine whether the user is calm.

[0927] The server decides the priority:

[0928] Priority: Low

[0929] Providing timely notification to nurses

[0930] The nurse begins to respond:

[0931] The nurse will contact the patient after other emergency procedures have been completed.

[0932] Sends a timestamp of when the response started to the server

[0933] Status report after completion of response:

[0934] The nurse completes the information provision and sends the end timestamp to the server.

[0935] The server logs the completion of the process and updates the non-emergency history.

[0936] This system will improve the work efficiency of nurses, enable them to respond in a way that takes patients' feelings into consideration, and increase patient satisfaction.

[0937] The processing flow will be explained below.

[0938] Step 1:

[0939] The user presses the nurse call button on the terminal. This sends a signal to start the nurse call. Specifically, the terminal waits for user input and detects that the button has been pressed.

[0940] Step 2:

[0941] The terminal acquires the user's identification information. The identification information is automatically read using, for example, a radio frequency identification tag or a barcode reader. This allows identification information such as a patient ID to be acquired.

[0942] Step 3:

[0943] The user speaks their inquiry or request to the terminal. For example, they can input messages such as "I have a stomachache" or "Please tell me when my next appointment is."

[0944] Step 4:

[0945] The device collects the audio data and temporarily stores it as an audio file, which is accompanied by a timestamp and patient identification information.

[0946] Step 5:

[0947] The device sends the voice data and identification information to the server. The data is encrypted and sent securely using the HTTPS protocol.

[0948] Step 6:

[0949] The server passes the received voice data and identification information to the generative AI, which then converts the voice data into text using voice recognition technology.

[0950] Step 7:

[0951] Generative artificial intelligence uses speech recognition technology to convert voice data into text, and then passes the converted text data to the next analysis stage.

[0952] Step 8:

[0953] Generative AI analyzes text data using natural language processing (NLP) techniques to assess the patient's needs and urgency, detecting keywords such as "pain," "fall," and "dyspnea."

[0954] Step 9:

[0955] The server passes the acquired text data to the emotion engine, which analyzes the voice tone, speech content, and speech rate to recognize the user's emotions.

[0956] Step 10:

[0957] An emotion engine analyzes voice tone and speaking rate to assess the user's emotion (e.g., anxious, calm, nervous, etc.).

[0958] Step 11:

[0959] The server integrates data from the emotion engine and generative artificial intelligence, and determines priorities based on the request content, urgency, and the user's emotional evaluation. For example, if anxiety is high, the urgency level is set high.

[0960] Step 12:

[0961] The server sends notifications to the nurse's terminal based on priority: high-priority requests are notified immediately, and medium- and low-priority requests are notified at an appropriate time.

[0962] Step 13:

[0963] The terminal (nurse's device) receives notifications from the server and notifies the nurse with audio and visual alerts, including the request, priority, and the user's emotional state.

[0964] Step 14:

[0965] The user (nurse) checks the notification and starts responding based on the specified priority. When the nurse starts responding, they input the start of response into the terminal.

[0966] Step 15:

[0967] The device sends a timestamp of when the response started to the server, and the server confirms that the response started and records it in a log.

[0968] Step 16:

[0969] The user (nurse) visits the patient and provides treatment. Specifically, the user checks the patient's symptoms and provides the necessary treatment.

[0970] Step 17:

[0971] When the user (nurse) completes the treatment, they input the completion of the treatment into the terminal. The terminal records a timestamp of the completion of the treatment and sends it to the server.

[0972] Step 18:

[0973] The server receives the timestamp of the response completion and updates the response history, thereby recording the nurse call processing status as completed.

[0974] With this processing flow, the nurse call system of the present invention not only efficiently processes patient requests and reduces the burden on nurses, but also enables prompt and appropriate responses that take into consideration the patient's feelings.

[0975] Example 2

[0976] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0977] Conventional nurse call systems have difficulty accurately grasping the level of urgency and emotional state of patients, resulting in delays in nurses' responses and an inability to assign appropriate priorities. As a result, patient satisfaction has decreased and nurses' workload has increased. To solve this, a system that can respond quickly and accurately while also taking into account the patient's emotional state is needed.

[0978] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0979] In this invention, the server includes means for analyzing voice data using generative artificial intelligence and evaluating the content and urgency of requests, means for analyzing the user's emotions using an emotion analysis engine and reflecting this in the urgency evaluation, and means for determining priorities based on the urgency. This enables responses that take into account not only the content of the patient's request but also their emotional state, supporting nurses in responding quickly and appropriately, improving patient satisfaction and reducing the workload of nurses.

[0980] A "user" is a patient or a patient's representative who uses the system to make a nurse call.

[0981] A "terminal" is a device that allows a user to initiate a nurse call, and is a device that has the role of collecting patient identification information and voice data and transmitting them to a server.

[0982] The "server" is a central processing unit that receives data sent from the terminal, analyzes it using generative artificial intelligence and an emotion analysis engine, and sends appropriate notifications to nurses.

[0983] "Generative AI" is an AI technology that analyzes voice data, converts it into text, and uses natural language processing technology to evaluate the content and urgency of requests.

[0984] An "emotion analysis engine" is a technology that analyzes the tone of voice, speech content, and speech rate to recognize the user's emotional state.

[0985] "Urgency" is an evaluation index that indicates the urgency of responding to a nurse call from a user, and is an important parameter for determining the priority of nurses' responses.

[0986] "Priority" is an index that indicates the priority of the response that the server determines for a nurse call, and indicates the order in which requests are responded to.

[0987] A "nurse" is a medical professional who is responsible for responding to nurse calls from users, receiving notifications via a terminal, and taking appropriate action.

[0988] A "notification" is a message sent from the server to the nurse's terminal, with a priority set based on the degree of urgency, to prompt the nurse to take action.

[0989] This invention incorporates generative artificial intelligence and an emotion analysis engine into a nurse call system to manage patient calls with high accuracy and support nurses' efficient responses. This system consists of five main components: the user, the terminal, the server, the nurse, and the emotion analysis engine. The specific functions of each component and the operation of the entire system are explained below.

[0990] User initiates nurse call

[0991] The user initiates contact using the nurse call button. This action initiates the nurse call reception process. The user can be the patient or their representative.

[0992] Terminal data collection and transmission

[0993] When the nurse call button is pressed, the terminal collects the patient's identification information and voice data. The identification information is automatically read using a radio frequency identification tag reader or barcode reader. The terminal then transmits this information to a server. This data transmission is encrypted using a secure protocol (e.g., HTTPS).

[0994] Server data analysis

[0995] The server receives the voice data sent from the device and passes it to a generative AI. The generative AI converts the voice data into text using speech recognition technology (e.g., Google Cloud Speech-to-Text). The server then analyzes the converted text using natural language processing technology (e.g., spaCy, BERT) to evaluate the content and urgency of the request.

[0996] Emotion analysis using an emotion analysis engine

[0997] The server passes the voice and text data to an emotion analysis engine (e.g., IBM Watson Tone Analyzer), which analyzes the voice tone, speech content, and speech rate to recognize the user's emotions. This allows the emotion analysis results to be reflected in the urgency assessment. For example, if the user feels highly anxious, the urgency can be set even higher.

[0998] Prioritizing and sending notifications

[0999] The server determines the priority of the nurse call based on the analysis results and sends notifications to the nurse's device accordingly. High-priority notifications are sent immediately, while medium- and low-priority notifications are sent at an appropriate time. Notifications include the request content, priority, and the user's emotional state.

[1000] Nurse's response and report

[1001] The user (nurse) checks the notification from the terminal and begins responding based on the priority. When the nurse starts responding, they input the start of response into the terminal, and the timestamp is sent to the server. When the response ends, the end timestamp is also sent from the terminal to the server. This causes the server to update the response history and complete the entire process.

[1002] Specific examples

[1003] Example 1: Emergency nurse call

[1004] User (patient) presses nurse call button:

[1005] Patient: "My stomach hurts, can you do something about it?"

[1006] The device sends the information to the server:

[1007] Patient ID: 12345

[1008] Audio data: "My stomach hurts, can you help me?"

[1009] The server analyzes the data using generative artificial intelligence:

[1010] Generative artificial intelligence converts voice data into text and analyzes the request using natural language processing technology.

[1011] Urgency: High

[1012] The server performs sentiment analysis using the sentiment analysis engine:

[1013] The sentiment analysis engine detects the user's anxiety and further increases the level of urgency.

[1014] The server decides the priority:

[1015] Priority: High

[1016] Immediate notification to nurses

[1017] The nurse begins:

[1018] A nurse immediately rushes to the patient's room and provides treatment.

[1019] Sends a timestamp of when the response started to the server

[1020] Status report after completion of response:

[1021] The nurse completes the procedure and sends the completion timestamp to the server.

[1022] The server logs the completion of the process and updates the emergency response history.

[1023] Example 2: Non-emergency nurse call

[1024] User (patient) presses nurse call button:

[1025] Patient: "What time is my next appointment?"

[1026] The device sends the information to the server:

[1027] Patient ID: 67890

[1028] Audio data: "What time is my next appointment?"

[1029] The server analyzes the data using generative artificial intelligence:

[1030] Generative artificial intelligence converts voice data into text and analyzes the request using natural language processing technology.

[1031] Urgency: Low

[1032] The server performs sentiment analysis using the sentiment analysis engine:

[1033] The sentiment analysis engine determines that the user is calm.

[1034] The server decides the priority:

[1035] Priority: Low

[1036] Providing timely notification to nurses

[1037] The nurse begins:

[1038] The nurse will contact the patient after other emergency procedures have been completed.

[1039] Sends a timestamp of when the response started to the server

[1040] Status report after completion of response:

[1041] The nurse completes the information provision and sends the end timestamp to the server.

[1042] The server logs the completion of the process and updates the non-emergency history.

[1043] By utilizing a generative AI model and an emotion analysis engine, this system enables quick and appropriate responses that take into account the patient's emotions, thereby reducing the burden on nurses and improving patient satisfaction.

[1044] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1045] Step 1:

[1046] The user presses the nurse call button.

[1047] Specific operation: The patient initiates contact by pressing the nurse call button next to the bed, which triggers the terminal to start accepting and processing the nurse call.

[1048] Input: User action (pressing nurse call button)

[1049] Output: Nurse call start signal

[1050] Step 2:

[1051] The terminal collects patient identification information and voice data.

[1052] What it does: The device's microphone records the audio and uses a radio frequency identification tag reader or barcode reader to read the patient's identification information.

[1053] Input: Nurse call start signal

[1054] Output: Audio data and identification information

[1055] Step 3:

[1056] The terminal transmits the collected data to the server.

[1057] Specific operation: The device encrypts the collected voice data and patient identification information using the HTTPS protocol and sends it to the server.

[1058] Input: Audio data and identification information

[1059] Output: Encrypted data packet

[1060] Step 4:

[1061] The server receives the voice data and passes it to the generative artificial intelligence.

[1062] Specific operation: Voice data and identification information are stored in a database, and the voice data is sent to generative artificial intelligence for voice recognition.

[1063] Input: Encrypted data packet

[1064] Output: Text data (speech recognition results)

[1065] Step 5:

[1066] The server analyzes the text data using natural language processing technology and evaluates the request content and urgency.

[1067] What it does: The server uses a natural language processing library (e.g., spaCy) to analyze the text data and extract the necessary information (e.g., "pain," "abdomen," "urgent," etc.).

[1068] Input: Text data

[1069] Output: Request details and urgency

[1070] Step 6:

[1071] The server passes the voice and text data to an emotion analysis engine to analyze the user's emotions.

[1072] Specific operation: Analyzes voice tone, speech content, and speech rate using an emotion analysis engine (e.g., IBM Watson Tone Analyzer) to recognize the user's emotions.

[1073] Input: Audio and text data

[1074] Output: Emotional state

[1075] Step 7:

[1076] The server reflects the emotion analysis results in the urgency assessment and determines the priority.

[1077] What you do: Reassess the urgency and prioritize it as high, medium, or low based on your emotional state and the request.

[1078] Input: Request, urgency, emotional state

[1079] Output: Priority

[1080] Step 8:

[1081] The server sends a notification to the nurse's terminal.

[1082] Specific behavior: The server generates notifications based on priority and sends them to the nurse's device. High priority notifications are sent immediately.

[1083] Input: Priorities, Needs, Emotional State

[1084] Output: Notification data

[1085] Step 9:

[1086] The user (nurse) receives the notification from the terminal and begins to respond.

[1087] Specific actions: A notification appears on the nurse's device, and an audio and visual alert is sent. The nurse checks the notification and begins to respond.

[1088] Input: Notification data

[1089] Output: Now supported

[1090] Step 10:

[1091] The terminal sends a timestamp of when the response started to the server and updates the response log.

[1092] Specific operation: The nurse presses the start response button on their device, the timestamp is sent to the server, and the log is updated.

[1093] Input: Start of response

[1094] Output: timestamp

[1095] Step 11:

[1096] When the user (nurse) has completed the treatment, the completion of the treatment is input into the terminal.

[1097] Specific action: The nurse completes the procedure and presses the "procedure completed" button on the terminal.

[1098] Input: Completed

[1099] Output: Completion signal

[1100] Step 12:

[1101] The device sends a timestamp of the completion to the server and updates the response history.

[1102] Specific operation: A completion timestamp is sent from the device to the server, and the action history is recorded as completed.

[1103] Input: Completion signal

[1104] Output: Completion timestamp and updated history

[1105] (Application example 2)

[1106] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1107] Conventional factory robot maintenance systems were sometimes slow to respond appropriately when an abnormality occurred, making it difficult to respond quickly to highly urgent events. Furthermore, there was a lack of a system that could automatically evaluate the nature and urgency of the abnormality and appropriately determine priorities. This resulted in an increased burden on maintenance staff and a decline in the overall factory's production efficiency, which became an issue.

[1108] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing voice data using generative artificial intelligence and evaluating the nature of the abnormality and the level of urgency, means for determining a priority based on the level of urgency, and means for sending a notification according to the priority to a staff member's terminal. This enables a prompt and appropriate response when an abnormality occurs in a robot in a factory, reducing the burden on maintenance staff and improving factory production efficiency.

[1109] A "user" is a person or device that accesses the system and reports an abnormality, and has the role of using a terminal to transmit identification information and voice data to the server.

[1110] "Terminal" means a device operated by a user to collect and transmit identification information and voice data, including computing devices such as smartphones and tablets.

[1111] "Identification information" is information for identifying a user or a robot, and is a unique identification code obtained by a radio frequency identification tag (RFID) or a barcode.

[1112] "Voice data" refers to voice information uttered by a user or a robot, and is data that is converted into text using voice recognition technology.

[1113] A "server" is a centralized computer system that receives data sent from devices, analyzes the data using generative artificial intelligence and emotion engines, and issues instructions for appropriate responses.

[1114] "Generative AI" is a type of AI technology that analyzes input voice data, converts it into text, and uses natural language processing technology to evaluate the content of requests and abnormalities.

[1115] "Natural language processing" is a technology that enables generative artificial intelligence to analyze text data, understand its meaning, and generate appropriate responses.

[1116] An "emotion engine" is a technology that analyzes the emotions of a user or situation from voice data or text data and reflects this in the assessment of urgency.

[1117] "Priority" is an evaluation criterion that classifies the priority of a response as "high," "medium," or "low" based on the results of analysis by the server.

[1118] "Notification" is information sent to staff terminals according to the priority determined by the server, and is received as an alert or message.

[1119] "Staff" refers to the people who receive notifications from the system and take appropriate action, including maintenance workers and nurses.

[1120] The present invention relates to a robot maintenance system used in a factory that enables a quick and appropriate response when an abnormality occurs. The present invention provides a system including the following main components:

[1121] 1. User abnormality notification

[1122] The user can use the terminal to report any abnormalities in the robot. When an abnormality occurs, the user can press a button on the terminal or report the abnormality by voice, which will trigger the system to notify the abnormality.

[1123] 2. Terminal data collection and transmission

[1124] The terminal collects the user's or robot's identification information and voice data and sends it to the server. The identification information is automatically read using a radio frequency identification tag (RFID) or barcode reader. The identification information and voice data acquired by the terminal are encrypted using a secure protocol (e.g., HTTPS) and sent to the server.

[1125] 3. Server Data Analysis

[1126] The server passes the received voice data to a generative artificial intelligence (AI) system, which converts it into text using speech recognition technology. The converted text data is then analyzed using natural language processing (NLP) technology to assess the nature of the anomaly and its urgency.

[1127] 4. Emotion analysis using an emotion engine

[1128] The server passes the analyzed text data to the emotion engine, which then reflects the emotion analysis in assessing the urgency of the anomaly. The emotion engine recognizes emotions from the voice tone, speech content, speech rate, or the robot's movement patterns, and adjusts the urgency accordingly.

[1129] 5. Prioritization

[1130] The server determines the priority of anomalies based on the analysis results of the generative artificial intelligence and emotion engine, and classifies the priority as "high," "medium," or "low."

[1131] 6. Notification to staff

[1132] The server sends notifications to the maintenance staff's terminals according to the determined priority. High-priority anomalies are notified immediately, while medium- and low-priority anomalies are notified at an appropriate time. The notification includes the nature of the anomaly, priority, and emotional state.

[1133] 7. Staff Response and Reporting

[1134] The user (maintenance staff) receives the notification from the terminal and begins to respond based on priority. When the staff member starts to respond, they input "response start" into the terminal. A timestamp of the response start is sent to the server and the response log is updated. When the response is completed, they input "response completion" into the terminal and a completion timestamp is sent to the server. The server receives the response completion timestamp, updates the response history, and records the abnormality processing status.

[1135] Specific examples

[1136] As example 1, we will introduce a scenario in which an emergency robot abnormality occurs.

[1137] Robot ID: robot_123

[1138] Error message: Critical failure, motor overheating.

[1139] Analyze the details of the error message with generative AI and use an emotion engine to assess urgency and display the emotional state.

[1140] The server analyzes this error message, rates it as high urgency, and sends an immediate notification to maintenance staff.

[1141] The above configuration enables prompt and appropriate response when an abnormality occurs in a robot in a factory, thereby reducing the burden on maintenance staff and improving factory production efficiency.

[1142] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1143] Step 1:

[1144] User abnormality notification begins

[1145] When an abnormality occurs in a robot in a factory, the user can report the abnormality by pressing a dedicated button on the terminal or by voice. The input is a voice message or button operation when an abnormality occurs. The output is a signal notifying the terminal of the abnormality. This causes the system to start receiving abnormality notifications.

[1146] Step 2:

[1147] Collection of identification information and voice data

[1148] The terminal automatically reads the user's or robot's identification information using an RFID or barcode reader, and also collects voice messages from the user as voice data. The input is the RFID tag or barcode and the voice message. The output is the identification information and voice data.

[1149] Step 3:

[1150] Sending data to the server

[1151] The device sends the collected identification information and voice data to the server using a secure protocol (e.g., HTTPS). The input is the identification information and voice data. The output is the data sent to the server.

[1152] Step 4:

[1153] Analysis of audio data

[1154] The server passes the received voice data to a generative AI, which converts the voice data into text using speech recognition technology. The input is the voice data, and the output is the converted text data. A generative AI model (e.g., GPT-4) is used for this process.

[1155] Step 5:

[1156] Text data analysis

[1157] The server analyzes the converted text data using a natural language processing (NLP) tool to evaluate the nature of the anomaly and its urgency. The input is the text data. The output is the analysis results, which include the nature of the anomaly and its urgency. An NLP tool (e.g., SpaCy) is used for this process.

[1158] Step 6:

[1159] Emotion Analysis

[1160] The server passes the analyzed text data to an emotion engine, which recognizes emotions from the voice tone and content of the speech. The input is text data. The output is the emotional state and urgency adjustment information as the result of emotion analysis. An emotion engine (e.g., IBM Watson Tone Analyzer) is used for this process.

[1161] Step 7:

[1162] Determining priorities

[1163] The server determines the priority of anomalies based on the analysis results of the generative artificial intelligence and emotion engine. The inputs are the details of the anomaly, the urgency, and the emotion analysis results. The output is the determined priority. Priorities are classified as "high," "medium," or "low."

[1164] Step 8:

[1165] Sending notifications to staff

[1166] The server sends a notification to the maintenance staff's terminal based on the determined priority. The input is the determined priority, the abnormality details, and the staff's emotional state. The output is a notification to the staff's terminal. The notification includes the abnormality details, priority, and emotional state.

[1167] Step 9:

[1168] Staff begins to respond

[1169] The user (maintenance staff) receives the notification and starts a response based on the priority. When the response starts, a timestamp of the response start is sent from the terminal to the server. The input is the notification and the start time, and the output is a response start log.

[1170] Step 10:

[1171] Response completion report

[1172] When the user (maintenance staff) completes the response, they input the response completion into the terminal and send a completion timestamp to the server. The input is the response completion time, and the output is a response completion log. The server receives the response completion timestamp, updates the response history, and records the abnormality processing status.

[1173] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1174] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1175] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1176] [Third embodiment]

[1177] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1178] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1179] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1180] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1181] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1182] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1183] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1184] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1185] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1186] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1187] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1188] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1189] This invention provides a method and system for incorporating generative artificial intelligence into a nurse call system to effectively manage patient contacts. This system works with the cooperation of users (patients), terminals, a server, and nurses.

[1190] System configuration

[1191] 1. User initiates nurse call

[1192] The user initiates contact by pressing the nurse call button on the terminal.

[1193] 2. Terminal data collection and transmission

[1194] The terminal obtains the user's identification information (eg, a radio frequency identification tag or a barcode) and collects the user's voice data.

[1195] The terminal transmits the acquired identification information and voice data to the server.

[1196] 3. Server Data Analysis

[1197] The server passes the received voice data to the generative artificial intelligence.

[1198] Generative AI uses speech recognition technology to convert voice data into text, and then uses natural language processing technology to analyze the text data, understand the content, and assess its urgency.

[1199] 4. Server Prioritization

[1200] The server evaluates the urgency of the nurse call and determines its priority based on the analysis results of the generative artificial intelligence.

[1201] 5. Sending notifications

[1202] The server sends notifications to the nurses' terminals based on the determined priority: high-priority requests are notified immediately, and low-priority requests are notified at an appropriate time.

[1203] 6. Nurse response and reporting

[1204] The user (nurse) receives the notification from the terminal and responds according to priority.

[1205] Nurses report their response status from their terminals to the server, and the progress and completion status of the response are updated.

[1206] Specific examples

[1207] Example 1: Emergency nurse call

[1208] User (patient) presses nurse call button:

[1209] Patient: "My stomach hurts, can you do something about it?"

[1210] The device sends the information to the server:

[1211] Patient ID: 12345

[1212] Audio data: "My stomach hurts, can you help me?"

[1213] The server analyzes the data using generative artificial intelligence:

[1214] Generative AI: Analyzes the content of voice data and determines "pain," "abdomen," and "urgency."

[1215] Urgency: High

[1216] The server decides the priority:

[1217] Priority: High

[1218] Immediate notification to nurses

[1219] The nurse begins to respond:

[1220] A nurse immediately rushes to the patient's room and provides treatment.

[1221] Sends a timestamp of when the response started to the server

[1222] Status report after completion of response:

[1223] The nurse completes the procedure and sends the end timestamp to the server.

[1224] The server logs the completion of the process and updates the emergency response history.

[1225] Example 2: Non-emergency nurse call

[1226] User (patient) presses nurse call button:

[1227] Patient: "What time is my next appointment?"

[1228] The device sends the information to the server:

[1229] Patient ID: 67890

[1230] Audio data: "What time is my next appointment?"

[1231] The server analyzes the data using generative artificial intelligence:

[1232] Generative artificial intelligence: Analyzes the content of voice data and determines "consultation time" and "information provision."

[1233] Urgency: Low

[1234] The server decides the priority:

[1235] Priority: Low

[1236] Providing timely notification to nurses

[1237] The nurse begins to respond:

[1238] The nurse will contact the patient after other emergency procedures have been completed.

[1239] Sends a timestamp of when the response started to the server

[1240] Status report after completion of response:

[1241] The nurse completes the information provision and sends the end timestamp to the server.

[1242] The server logs the completion of the process and updates the non-emergency history.

[1243] In this way, the system of the present invention is designed to improve nurse efficiency and increase patient satisfaction.

[1244] The processing flow will be explained below.

[1245] Step 1:

[1246] The user presses the nurse call button on the terminal, which sends out a signal to start a nurse call. Specifically, the terminal is waiting to receive user input and detects that the button has been pressed.

[1247] Step 2:

[1248] The terminal acquires the patient's identification information, which is automatically read using a radio frequency identification tag or barcode reader.

[1249] Step 3:

[1250] The user communicates their inquiry or request to the terminal by voice. For example, they can input a message such as "My stomach hurts."

[1251] Step 4:

[1252] The device collects the audio data and temporarily stores it as an audio file, which is accompanied by a timestamp and patient identification information.

[1253] Step 5:

[1254] The device sends the voice data and identification information to the server using the secure HTTPS protocol, and the data is encrypted.

[1255] Step 6:

[1256] The server passes the received voice data to the generative AI, which has a dedicated module for analyzing the voice data.

[1257] Step 7:

[1258] Generative artificial intelligence uses speech recognition technology to convert audio data into text, which is then passed on to the next processing stage for analysis.

[1259] Step 8:

[1260] Generative AI analyzes text data using natural language processing (NLP) techniques to assess the patient's needs and urgency, detecting keywords such as "pain," "fall," and "dyspnea."

[1261] Step 9:

[1262] Based on the results of the generative AI analysis, the server evaluates the urgency and determines the priority of the response, which is classified as "high," "medium," or "low."

[1263] Step 10:

[1264] The server sends notifications to the nurse's terminal according to priority. High-priority requests are notified immediately, while medium- and low-priority requests are notified at an appropriate time.

[1265] Step 11:

[1266] The terminal (nurse's device) receives notifications from the server and notifies the nurse with audio and visual alerts, including the request and priority.

[1267] Step 12:

[1268] The user (nurse) checks the notification and starts responding based on the specified priority. When the nurse starts responding, they input the start of response into the terminal.

[1269] Step 13:

[1270] The device sends a timestamp of when the response started to the server, and the server confirms that the response started and records it in a log.

[1271] Step 14:

[1272] The user (nurse) visits the patient and responds. Depending on the response, necessary measures are taken.

[1273] Step 15:

[1274] When the user (nurse) completes the treatment, they input the completion of the treatment into the terminal. The terminal sends a timestamp of the completion of the treatment to the server.

[1275] Step 16:

[1276] The server receives the timestamp of the response completion and updates the response history, thereby recording the nurse call processing status as completed in the log.

[1277] This processing flow allows the nurse call system to efficiently process patient requests, reducing the burden on nurses and improving patient satisfaction.

[1278] Example 1

[1279] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1280] In conventional nurse call systems, nurses may not be able to respond appropriately to calls from patients with different levels of urgency. This makes it difficult to respond quickly to calls with high urgency, which can lead to a decrease in patient safety and satisfaction. Furthermore, nurses may not be able to perform their work efficiently, which can result in excessive burdens. Therefore, the present invention aims to provide a system that effectively manages calls from patients, reducing the workload of nurses while enabling them to respond quickly and accurately.

[1281] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1282] In this invention, the server includes a means for a user to initiate a nurse call on a terminal, a means for the terminal to send the user's identification information and voice data to the server, a means for the server to analyze the voice data using generative artificial intelligence and evaluate the request content and urgency, a means for the server to determine the priority of the nurse call based on the analysis results, a means for the server to send a notification according to the urgency to the nurse's terminal, a means for the nurse to report the response status to the server via the terminal, and a means for the server to record the completion of processing in a log and update the response history. This enables a quick and accurate response that takes the urgency into consideration and reduces the workload of nurses.

[1283] "User" refers to the person who operates the terminal that initiates the nurse call.

[1284] "Terminal" refers to a device that acquires user identification information and voice data and transmits them to a server.

[1285] "Identification information" is unique information for identifying a user, and is obtained using, for example, a radio frequency identification tag or a barcode.

[1286] "Voice data" refers to data including what the user said when calling the nurse.

[1287] "Server" refers to a device or system that receives data sent from a terminal and analyzes it using generative artificial intelligence.

[1288] "Generative artificial intelligence" refers to an artificial intelligence model that uses voice recognition technology and natural language processing technology to analyze voice data.

[1289] "Voice recognition technology" refers to technology that converts voice data into text data.

[1290] "Natural language processing technology" refers to technology for analyzing text data and understanding its content.

[1291] "Urgency" refers to an indicator that evaluates how urgent the content of a nurse call is.

[1292] "Priority" refers to the criteria for determining the order of response based on the urgency of the nurse call.

[1293] "Notification" refers to a message that sends information about the nurse call and the level of urgency to the nurse's terminal.

[1294] "Caregiver" refers to a medical professional who takes action based on the notification sent.

[1295] "Response status" refers to the progress of a nurse's response to a nurse call.

[1296] "Log" refers to data that a server uses to record the completion of processing and response history.

[1297] This invention provides a method and system for incorporating generative artificial intelligence into a nurse call system to effectively manage calls from patients. This system works through the cooperation of users (patients), terminals, a server, and caregivers (nurses).

[1298] The system is implemented using the following hardware and software.

[1299] Terminal: A device that acquires a user's identification information and voice data and transmits them to a server. Specifically, a smart device equipped with a radio frequency identification tag reader or barcode reader is used.

[1300] Server: A device that uses generative artificial intelligence to process and analyze data received from the terminal, and sends notifications to the caregiver's terminal based on the determined priority. Specific implementation examples include cloud servers and on-premise servers.

[1301] Generative artificial intelligence: Software that analyzes audio data using speech recognition technologies (e.g., Google Cloud Speech-to-Text) and natural language processing technologies (e.g., Google Cloud Natural Language API).

[1302] This system allows users (patients) to quickly call a nurse in an emergency, and nurses can respond efficiently by receiving notifications according to priority.

[1303] The specific operation flow is shown below.

[1304] 1. User (patient) initiates nurse call

[1305] The user presses the nurse call button on the terminal to initiate a call. For example, a patient presses the nurse call button when they feel a stomach ache.

[1306] 2. Terminal data collection and transmission

[1307] The terminal acquires the user's identification information (e.g., radio frequency identification tag or barcode) and collects the user's voice data. For example, it reads "Patient ID: 12345" and sends the voice data "My stomach hurts. Can you do something about it?" to the server.

[1308] 3. Server Data Analysis

[1309] The server passes the received voice data to a generative AI. The generative AI converts the voice data into text using speech recognition technology, and then analyzes the text data using natural language processing technology to understand the content and assess the urgency. For example, the voice data "My stomach hurts. Can you do something?" would be analyzed as "pain," "abdomen," and "high urgency."

[1310] 4. Server Prioritization

[1311] The server evaluates the urgency of the nurse call based on the analysis results of the generative AI and determines the priority. For example, if the urgency is evaluated as "high," the priority will be set to "high."

[1312] 5. Sending notifications

[1313] The server sends a notification to the nurse's terminal based on the determined priority. For example, if the priority is "High," an immediate notification is sent. The notification content may include something like, "Please go immediately to the room of patient ID: 12345. The patient is complaining of stomach pain."

[1314] 6. Caregiver response and reporting

[1315] The user (caregiver) receives the notification from the terminal and responds according to priority. For example, the nurse who received the notification rushes to the patient's room. The caregiver reports the response status from the terminal to the server, records the completion of the process in a log, and updates the response history. For example, timestamps at the start and end of the response are sent to the server.

[1316] Specific examples

[1317] Example 1: Emergency nurse call

[1318] User (patient) presses nurse call button:

[1319] Patient: "My stomach hurts, can you do something about it?"

[1320] The device sends the information to the server:

[1321] Patient ID: 12345

[1322] Audio data: "My stomach hurts, can you help me?"

[1323] The server analyzes the data using generative artificial intelligence:

[1324] Generative AI: Analyzes the content of the voice data and determines whether the condition is "painful," "abdominal," or "high urgency."

[1325] Urgency: High

[1326] The server decides the priority:

[1327] Priority: High

[1328] Immediate notification to caregivers

[1329] Caregiver initiates response:

[1330] A caregiver immediately goes to the patient's room and provides treatment.

[1331] Sends a timestamp of when the response started to the server

[1332] Status report after completion of response:

[1333] The caregiver completes the procedure and sends the completion timestamp to the server.

[1334] The server logs the completion of the process and updates the emergency response history.

[1335] Example 2: Non-emergency nurse call

[1336] User (patient) presses nurse call button:

[1337] Patient: "What time is my next appointment?"

[1338] The device sends the information to the server:

[1339] Patient ID: 67890

[1340] Audio data: "What time is my next appointment?"

[1341] The server analyzes the data using generative artificial intelligence:

[1342] Generative artificial intelligence: Analyzes the content of voice data and determines "consultation time" and "information provision."

[1343] Urgency: Low

[1344] The server decides the priority:

[1345] Priority: Low

[1346] Providing timely notification to caregivers

[1347] Caregiver initiates response:

[1348] After other emergency procedures are completed, the caregiver contacts the patient.

[1349] Sends a timestamp of when the response started to the server

[1350] Status report after completion of response:

[1351] The caregiver completes the information submission and sends an end timestamp to the server.

[1352] The server logs the completion of the process and updates the non-emergency response history.

[1353] In this manner, the system of the present invention is designed to improve caregiver efficiency and increase patient satisfaction.

[1354] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1355] Step 1:

[1356] User (patient) initiates nurse call

[1357] This step begins when the user presses the nurse call button on the terminal. Specifically, the patient presses the button when they feel a stomach ache.

[1358] Input: User (patient) presses nurse call button

[1359] Output: Nurse call start signal

[1360] Step 2:

[1361] Terminal data collection and transmission

[1362] Upon receiving the nurse call start signal, the terminal simultaneously acquires the user's identification information (e.g., radio frequency identification tag or barcode) and collects the user's voice data. This data is then sent to the server. Specifically, the terminal reads "Patient ID: 12345" and sends the voice data "My stomach hurts. Can you do something about this?" to the server.

[1363] Input: Nurse call start signal, user identification information, voice data

[1364] Output: Sending a data packet containing the identification information and audio data to the server

[1365] Step 3:

[1366] Receiving data from the server and preparing for analysis

[1367] The server receives the data packets sent from the terminal and prepares them for analysis. In this step, preprocessing is performed to pass the received data to the generative artificial intelligence.

[1368] Input: Data packet containing identification information and voice data

[1369] Output: Data ready for analysis

[1370] Step 4:

[1371] Data analysis using generative artificial intelligence

[1372] The server passes the prepared data to the generative AI, which analyzes the voice data. The generative AI uses speech recognition technology to convert the voice data into text data, and then uses natural language processing technology to analyze the text data and evaluate the request's content and urgency. For example, it analyzes the voice data "My stomach hurts. Can you do something about this?" and determines that the request is "pain," "abdomen," and "high urgency."

[1373] Input: Data ready for analysis

[1374] Output: Text data and a severity rating based on that text

[1375] Step 5:

[1376] Server Prioritization

[1377] The server evaluates the urgency of the nurse call based on the analysis results of the generative artificial intelligence and determines the priority based on that evaluation. For example, if the urgency is determined to be "high," the priority is set to "high."

[1378] Input: Text data and urgency assessment using generative AI

[1379] Output: Priority determination

[1380] Step 6:

[1381] Sending notifications

[1382] The server then sends a notification to the nurse's terminal based on the determined priority. For example, if the priority is "High," it will immediately send a notification stating, "Please go immediately to the room of patient ID: 12345. The patient is complaining of stomach pain."

[1383] Input: Priority determination

[1384] Output: Notification to caregiver's terminal

[1385] Step 7:

[1386] Caregiver response and reporting

[1387] The user (caregiver) receives the notification from the terminal and responds according to the priority. For example, the caregiver who receives the notification rushes to the patient's room.

[1388] Input: Notification to caregiver's terminal

[1389] Output: Action to start the response

[1390] The nurse then reports the treatment status from the terminal to the server. The progress and completion status of the treatment are recorded using timestamps. Specifically, at the start and end of the treatment, the nurse presses the "Start Treatment" and "Complete Treatment" buttons to send timestamps to the server.

[1391] Input: Response start and completion timestamps

[1392] Output: Updates the corresponding log to the server

[1393] This enables the entire system to respond quickly and appropriately, taking into account the level of urgency, reducing the workload of caregivers and increasing patient satisfaction.

[1394] (Application example 1)

[1395] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1396] Existing security systems often fail to properly detect and respond to suspicious activities and emergencies, increasing the burden on security guards. In particular, systems lack mechanisms for proper prioritization and immediate notification in situations requiring rapid response based on urgency. Furthermore, even in situations where rapid backup is required in an emergency, the time lag in manual communication is a problem.

[1397] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1398] In this invention, the server includes means for acquiring audio or video data in real time and detecting suspicious activity or emergency situations using generative artificial intelligence, means for automatically making additional backup requests in the event of an emergency, and means for analyzing the acquired audio data and evaluating the request content and urgency, thereby enabling rapid detection and response to suspicious activity or emergency situations, and rapid backup requests in the event of an emergency.

[1399] A "user" is a human entity that uses a terminal to initiate a call.

[1400] A "terminal" is an electronic device that acquires voice data and identification information and transmits them to a server.

[1401] "Identification information" refers to information that identifies a particular user or patient, and includes radio frequency identification tags and bar codes.

[1402] A "server" is a computer system that processes and manages data on a network.

[1403] "Generative AI" is an AI technology that analyzes voice and text data to understand the content and assess its urgency.

[1404] "Voice data" is data containing voice information uttered by a user.

[1405] "Urgency" is a measure of the importance and urgency of a request or event to be handled.

[1406] "Priority" refers to the order in which events or requests to be addressed are ranked according to their importance.

[1407] "Notification" is the act of informing a terminal of a specific situation by sending information from a server to the terminal.

[1408] "Response status" is information that indicates the progress and results of actions taken in response to requests or events.

[1409] "Audio or video data" refers to data that includes audio and video information captured in real time.

[1410] "Suspicious activity" is any suspicious behavior, including fraudulent or unusual actions or movements.

[1411] An "emergency" is an urgent situation requiring immediate action.

[1412] A "backup request" is a request for additional assistance or response in an emergency.

[1413] The system for realizing this invention is mainly composed of three elements: a user, a terminal, and a server. A specific embodiment of this system will be described below.

[1414] The basic configuration of the system is that the user initiates a call using their device, which then sends identification information and voice data to the server. The server uses generative artificial intelligence to analyze the voice data and evaluate the request content and urgency. Based on the analysis results, the server determines a priority according to the urgency and sends a notification to the responder's device. The responder receives the notification and reports the response status to the server, improving overall work efficiency.

[1415] The server also captures audio and video data in real time and uses generative artificial intelligence to detect suspicious activity or emergencies. If suspicious activity or an emergency is detected, the server immediately assesses the severity and automatically requests additional backups if necessary.

[1416] Hardware and Software Use

[1417] 1. Device:

[1418] Wearable devices such as smartphones, smart glasses, and head-mounted displays will be used.

[1419] It is equipped with a microphone and a camera for capturing audio and video data.

[1420] 2. Software:

[1421] Speech Recognition: Use the SpeechRecognition library to convert voice data into text.

[1422] Sending data: Use the requests library to send data from the terminal to the server.

[1423] Generative AI: Using artificial intelligence models, such as GPT-3, to perform advanced analysis.

[1424] Specific examples

[1425] For example, imagine a suspicious individual appears while a security guard at a shopping mall is patrolling the area wearing smart glasses. If the suspicious individual utters a clear indication of fraudulent activity, such as "I'm going to break the camera," the microphone in the smart glasses captures the audio and immediately sends it to the server. The server then uses generative artificial intelligence to analyze the audio and evaluate whether or not there is fraudulent activity and the urgency of the situation. It then determines a priority and sends an immediate notification to the security guard's device. The guard who receives this notification can respond quickly and report the response status to the server in real time, thereby improving the overall security level.

[1426] Prompt Sentence Examples

[1427] Dear customer, please tell us more about your current situation. Is it an emergency? Do you need assistance?

[1428] Based on these specific examples, this system can realize a swift and appropriate response according to the level of urgency, which will significantly improve the security of facilities and locations, enabling effective measures to be taken in situations where a rapid response is required.

[1429] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1430] Step 1:

[1431] A user initiates a call on a terminal. The input is the user pressing the call button, and the terminal starts capturing audio. The output is the captured audio data and the user's identification information.

[1432] Step 2:

[1433] The terminal sends the user's identification information and voice data to the server. The input is the voice data and identification information acquired by the terminal, which are then sent to the server via the network. The output is the voice data and identification information passed to the server.

[1434] Step 3:

[1435] The server analyzes the voice data using a generative artificial intelligence model. The voice data is sent as input, and the generative artificial intelligence is used to convert the voice into text and analyze the content. Specifically, the voice data is converted into text data using voice recognition technology, and then the content of the text is understood using natural language processing technology, and the request content and urgency are evaluated. The analysis results are generated as output.

[1436] Step 4:

[1437] Based on the analysis results, the server evaluates the request content and urgency and determines the priority. The input is the analysis results from the generative AI, and based on this, the urgency is evaluated and priorities are set. For example, an evaluation such as "high urgency" or "low urgency" is made. A priority list is generated as the output.

[1438] Step 5:

[1439] The server sends notifications to the responder's device according to the priority. The input is a priority list, and notifications are sent to the responder's device based on this information. If the priority is high, immediate notification is sent, and if the priority is low, timely notification is sent. The output is a notification that arrives on the responder's device.

[1440] Step 6:

[1441] The responder reports the response status to the server via the terminal. The input includes details of the response performed by the responder and the progress status, which are reported via the terminal. Specifically, the timestamps for when the response started and when it was completed are reported. The output is a record of the response status on the server.

[1442] Step 7:

[1443] The server acquires audio or video data in real time and uses generative artificial intelligence to detect suspicious activity or emergencies. Real-time audio or video data is input, which is then analyzed by generative artificial intelligence. Specifically, voice recognition and video analysis are performed to detect suspicious sounds and behavior. The output is a detection result of suspicious activity or an emergency.

[1444] Step 8:

[1445] In an emergency, the server automatically requests additional backups. The input is the detection result of suspicious activity or an emergency, and the backup request is made according to the level of urgency. Specifically, if the level of urgency is high, a request for assistance is automatically sent to additional responders. The output is a backup request sent to the responder.

[1446] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1447] This invention relates to a system that incorporates generative artificial intelligence and an emotion engine into a nurse call system to accurately manage patient contact and support nurses' efficient responses. This system consists of five main components: the user (patient), terminal, server, nurse, and emotion engine.

[1448] System configuration

[1449] 1. User initiates nurse call

[1450] The user initiates contact by pressing the nurse call button on the terminal. This action causes the system to start the nurse call reception process.

[1451] 2. Terminal data collection and transmission

[1452] The terminal obtains the user's identification information and collects the voice data, which is automatically read using a radio frequency identification tag or bar code reader.

[1453] The device sends the acquired identification information and voice data to the server. This data is encrypted using a secure protocol (e.g., HTTPS) and then transmitted.

[1454] 3. Server Data Analysis

[1455] The server passes the received voice data to the generative AI, which converts the voice data into text using voice recognition technology.

[1456] The server receives the converted text data, analyzes it using natural language processing (NLP) technology, and evaluates the content and urgency of the request.

[1457] 4. Emotion analysis using an emotion engine

[1458] The server passes the acquired voice and text data to the emotion engine, which analyzes the voice tone, speech content, and speech rate to recognize the user's emotions.

[1459] The server reflects the recognized emotion in the assessment of urgency and adjusts the priority as necessary.

[1460] 5. Server Prioritization

[1461] The server determines the priority of nurse call requests based on the analysis results of the generative artificial intelligence and emotion engine. Priority is classified as "high," "medium," or "low."

[1462] 6. Sending Notifications

[1463] The server sends notifications to the nurses' terminals based on the determined priority. High-priority requests are notified immediately, and medium- and low-priority requests are notified at an appropriate time.

[1464] The user (nurse) receives notifications from the device and confirms them with audio and visual alerts, which include the request, priority, and the user's emotional state.

[1465] 7. Nurse response and reporting

[1466] The user (nurse) checks the notification on the terminal and starts responding based on the priority. When the nurse starts responding, they input the start of response on the terminal.

[1467] The terminal sends a timestamp of when the response started to the server, and the response log is updated.

[1468] When the user (nurse) completes the treatment, they input the completion of the treatment into the terminal. The terminal records a timestamp of the completion of the treatment and sends it to the server.

[1469] The server receives the timestamp of the response completion and updates the response history, thereby recording the nurse call processing status as completed.

[1470] Specific examples

[1471] Example 1: Emergency nurse call

[1472] User (patient) presses nurse call button:

[1473] Patient: "My stomach hurts, can you do something about it?"

[1474] The device sends the information to the server:

[1475] Patient ID: 12345

[1476] Audio data: "My stomach hurts, can you help me?"

[1477] The server analyzes the data using generative artificial intelligence:

[1478] Generative AI: Analyzes the content of voice data and determines "pain," "abdomen," and "urgency."

[1479] Urgency: High

[1480] The server performs sentiment analysis using the emotion engine:

[1481] Emotion Engine: Analyzes voice tone and speech rate to detect the user's anxiety.

[1482] Detecting anxiety will set the level of urgency even higher.

[1483] The server decides the priority:

[1484] Priority: High

[1485] Immediate notification to nurses

[1486] The nurse begins to respond:

[1487] A nurse immediately rushes to the patient's room and provides treatment.

[1488] Sends a timestamp of when the response started to the server

[1489] Status report after completion of response:

[1490] The nurse completes the procedure and sends the end timestamp to the server.

[1491] The server logs the completion of the process and updates the emergency response history.

[1492] Example 2: Non-emergency nurse call

[1493] User (patient) presses nurse call button:

[1494] Patient: "What time is my next appointment?"

[1495] The device sends the information to the server:

[1496] Patient ID: 67890

[1497] Audio data: "What time is my next appointment?"

[1498] The server analyzes the data using generative artificial intelligence:

[1499] Generative artificial intelligence: Analyzes the content of voice data and determines "consultation time" and "information provision."

[1500] Urgency: Low

[1501] The server performs sentiment analysis using the emotion engine:

[1502] Emotion engine: Analyzes speech content and tone to determine whether the user is calm.

[1503] The server decides the priority:

[1504] Priority: Low

[1505] Providing timely notification to nurses

[1506] The nurse begins to respond:

[1507] The nurse will contact the patient after other emergency procedures have been completed.

[1508] Sends a timestamp of when the response started to the server

[1509] Status report after completion of response:

[1510] The nurse completes the information provision and sends the end timestamp to the server.

[1511] The server logs the completion of the process and updates the non-emergency history.

[1512] This system will improve the work efficiency of nurses, enable them to respond in a way that takes patients' feelings into consideration, and increase patient satisfaction.

[1513] The processing flow will be explained below.

[1514] Step 1:

[1515] The user presses the nurse call button on the terminal. This sends a signal to start the nurse call. Specifically, the terminal waits for user input and detects that the button has been pressed.

[1516] Step 2:

[1517] The terminal acquires the user's identification information. The identification information is automatically read using, for example, a radio frequency identification tag or a barcode reader. This allows identification information such as a patient ID to be acquired.

[1518] Step 3:

[1519] The user speaks their inquiry or request to the terminal. For example, they can input messages such as "I have a stomachache" or "Please tell me when my next appointment is."

[1520] Step 4:

[1521] The device collects the audio data and temporarily stores it as an audio file, which is accompanied by a timestamp and patient identification information.

[1522] Step 5:

[1523] The device sends the voice data and identification information to the server. The data is encrypted and sent securely using the HTTPS protocol.

[1524] Step 6:

[1525] The server passes the received voice data and identification information to the generative AI, which then converts the voice data into text using voice recognition technology.

[1526] Step 7:

[1527] Generative artificial intelligence uses speech recognition technology to convert voice data into text, and then passes the converted text data to the next analysis stage.

[1528] Step 8:

[1529] Generative AI analyzes text data using natural language processing (NLP) techniques to assess the patient's needs and urgency, detecting keywords such as "pain," "fall," and "dyspnea."

[1530] Step 9:

[1531] The server passes the acquired text data to the emotion engine, which analyzes the voice tone, speech content, and speech rate to recognize the user's emotions.

[1532] Step 10:

[1533] An emotion engine analyzes voice tone and speaking rate to assess the user's emotion (e.g., anxious, calm, nervous, etc.).

[1534] Step 11:

[1535] The server integrates data from the emotion engine and generative artificial intelligence, and determines priorities based on the request content, urgency, and the user's emotional evaluation. For example, if anxiety is high, the urgency level is set high.

[1536] Step 12:

[1537] The server sends notifications to the nurse's terminal based on priority: high-priority requests are notified immediately, and medium- and low-priority requests are notified at an appropriate time.

[1538] Step 13:

[1539] The terminal (nurse's device) receives notifications from the server and notifies the nurse with audio and visual alerts, including the request, priority, and the user's emotional state.

[1540] Step 14:

[1541] The user (nurse) checks the notification and starts responding based on the specified priority. When the nurse starts responding, they input the start of response into the terminal.

[1542] Step 15:

[1543] The device sends a timestamp of when the response started to the server, and the server confirms that the response started and records it in a log.

[1544] Step 16:

[1545] The user (nurse) visits the patient and provides treatment. Specifically, the user checks the patient's symptoms and provides the necessary treatment.

[1546] Step 17:

[1547] When the user (nurse) completes the treatment, they input the completion of the treatment into the terminal. The terminal records a timestamp of the completion of the treatment and sends it to the server.

[1548] Step 18:

[1549] The server receives the timestamp of the response completion and updates the response history, thereby recording the nurse call processing status as completed.

[1550] With this processing flow, the nurse call system of the present invention not only efficiently processes patient requests and reduces the burden on nurses, but also enables prompt and appropriate responses that take into consideration the patient's feelings.

[1551] Example 2

[1552] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1553] Conventional nurse call systems have difficulty accurately grasping the level of urgency and emotional state of patients, resulting in delays in nurses' responses and an inability to assign appropriate priorities. As a result, patient satisfaction has decreased and nurses' workload has increased. To solve this, a system that can respond quickly and accurately while also taking into account the patient's emotional state is needed.

[1554] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1555] In this invention, the server includes means for analyzing voice data using generative artificial intelligence and evaluating the content and urgency of requests, means for analyzing the user's emotions using an emotion analysis engine and reflecting this in the urgency evaluation, and means for determining priorities based on the urgency. This enables responses that take into account not only the content of the patient's request but also their emotional state, supporting nurses in responding quickly and appropriately, improving patient satisfaction and reducing the workload of nurses.

[1556] A "user" is a patient or a patient's representative who uses the system to make a nurse call.

[1557] A "terminal" is a device that allows a user to initiate a nurse call, and is a device that has the role of collecting patient identification information and voice data and transmitting them to a server.

[1558] The "server" is a central processing unit that receives data sent from the terminal, analyzes it using generative artificial intelligence and an emotion analysis engine, and sends appropriate notifications to nurses.

[1559] "Generative AI" is an AI technology that analyzes voice data, converts it into text, and uses natural language processing technology to evaluate the content and urgency of requests.

[1560] An "emotion analysis engine" is a technology that analyzes the tone of voice, speech content, and speech rate to recognize the user's emotional state.

[1561] "Urgency" is an evaluation index that indicates the urgency of responding to a nurse call from a user, and is an important parameter for determining the priority of nurses' responses.

[1562] "Priority" is an index that indicates the priority of the response that the server determines for a nurse call, and indicates the order in which requests are responded to.

[1563] A "nurse" is a medical professional who is responsible for responding to nurse calls from users, receiving notifications via a terminal, and taking appropriate action.

[1564] A "notification" is a message sent from the server to the nurse's terminal, with a priority set based on the degree of urgency, to prompt the nurse to take action.

[1565] This invention incorporates generative artificial intelligence and an emotion analysis engine into a nurse call system to manage patient calls with high accuracy and support nurses' efficient responses. This system consists of five main components: the user, the terminal, the server, the nurse, and the emotion analysis engine. The specific functions of each component and the operation of the entire system are explained below.

[1566] User initiates nurse call

[1567] The user initiates contact using the nurse call button. This action initiates the nurse call reception process. The user can be the patient or their representative.

[1568] Terminal data collection and transmission

[1569] When the nurse call button is pressed, the terminal collects the patient's identification information and voice data. The identification information is automatically read using a radio frequency identification tag reader or barcode reader. The terminal then transmits this information to a server. This data transmission is encrypted using a secure protocol (e.g., HTTPS).

[1570] Server data analysis

[1571] The server receives the voice data sent from the device and passes it to a generative AI. The generative AI converts the voice data into text using speech recognition technology (e.g., Google Cloud Speech-to-Text). The server then analyzes the converted text using natural language processing technology (e.g., spaCy, BERT) to evaluate the content and urgency of the request.

[1572] Emotion analysis using an emotion analysis engine

[1573] The server passes the voice and text data to an emotion analysis engine (e.g., IBM Watson Tone Analyzer), which analyzes the voice tone, speech content, and speech rate to recognize the user's emotions. This allows the emotion analysis results to be reflected in the urgency assessment. For example, if the user feels highly anxious, the urgency can be set even higher.

[1574] Prioritizing and sending notifications

[1575] The server determines the priority of the nurse call based on the analysis results and sends notifications to the nurse's device accordingly. High-priority notifications are sent immediately, while medium- and low-priority notifications are sent at an appropriate time. Notifications include the request content, priority, and the user's emotional state.

[1576] Nurse's response and report

[1577] The user (nurse) checks the notification from the terminal and begins responding based on the priority. When the nurse starts responding, they input the start of response into the terminal, and the timestamp is sent to the server. When the response ends, the end timestamp is also sent from the terminal to the server. This causes the server to update the response history and complete the entire process.

[1578] Specific examples

[1579] Example 1: Emergency nurse call

[1580] User (patient) presses nurse call button:

[1581] Patient: "My stomach hurts, can you do something about it?"

[1582] The device sends the information to the server:

[1583] Patient ID: 12345

[1584] Audio data: "My stomach hurts, can you help me?"

[1585] The server analyzes the data using generative artificial intelligence:

[1586] Generative artificial intelligence converts voice data into text and analyzes the request using natural language processing technology.

[1587] Urgency: High

[1588] The server performs sentiment analysis using the sentiment analysis engine:

[1589] The sentiment analysis engine detects the user's anxiety and further increases the level of urgency.

[1590] The server decides the priority:

[1591] Priority: High

[1592] Immediate notification to nurses

[1593] The nurse begins:

[1594] A nurse immediately rushes to the patient's room and provides treatment.

[1595] Sends a timestamp of when the response started to the server

[1596] Status report after completion of response:

[1597] The nurse completes the procedure and sends the completion timestamp to the server.

[1598] The server logs the completion of the process and updates the emergency response history.

[1599] Example 2: Non-emergency nurse call

[1600] User (patient) presses nurse call button:

[1601] Patient: "What time is my next appointment?"

[1602] The device sends the information to the server:

[1603] Patient ID: 67890

[1604] Audio data: "What time is my next appointment?"

[1605] The server analyzes the data using generative artificial intelligence:

[1606] Generative artificial intelligence converts voice data into text and analyzes the request using natural language processing technology.

[1607] Urgency: Low

[1608] The server performs sentiment analysis using the sentiment analysis engine:

[1609] The sentiment analysis engine determines that the user is calm.

[1610] The server decides the priority:

[1611] Priority: Low

[1612] Providing timely notification to nurses

[1613] The nurse begins:

[1614] The nurse will contact the patient after other emergency procedures have been completed.

[1615] Sends a timestamp of when the response started to the server

[1616] Status report after completion of response:

[1617] The nurse completes the information provision and sends the end timestamp to the server.

[1618] The server logs the completion of the process and updates the non-emergency history.

[1619] By utilizing a generative AI model and an emotion analysis engine, this system enables quick and appropriate responses that take into account the patient's emotions, thereby reducing the burden on nurses and improving patient satisfaction.

[1620] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1621] Step 1:

[1622] The user presses the nurse call button.

[1623] Specific operation: The patient initiates contact by pressing the nurse call button next to the bed, which triggers the terminal to start accepting and processing the nurse call.

[1624] Input: User action (pressing nurse call button)

[1625] Output: Nurse call start signal

[1626] Step 2:

[1627] The terminal collects patient identification information and voice data.

[1628] What it does: The device's microphone records the audio and uses a radio frequency identification tag reader or barcode reader to read the patient's identification information.

[1629] Input: Nurse call start signal

[1630] Output: Audio data and identification information

[1631] Step 3:

[1632] The terminal transmits the collected data to the server.

[1633] Specific operation: The device encrypts the collected voice data and patient identification information using the HTTPS protocol and sends it to the server.

[1634] Input: Audio data and identification information

[1635] Output: Encrypted data packet

[1636] Step 4:

[1637] The server receives the voice data and passes it to the generative artificial intelligence.

[1638] Specific operation: Voice data and identification information are stored in a database, and the voice data is sent to generative artificial intelligence for voice recognition.

[1639] Input: Encrypted data packet

[1640] Output: Text data (speech recognition results)

[1641] Step 5:

[1642] The server analyzes the text data using natural language processing technology and evaluates the request content and urgency.

[1643] What it does: The server uses a natural language processing library (e.g., spaCy) to analyze the text data and extract the necessary information (e.g., "pain," "abdomen," "urgent," etc.).

[1644] Input: Text data

[1645] Output: Request details and urgency

[1646] Step 6:

[1647] The server passes the voice and text data to an emotion analysis engine to analyze the user's emotions.

[1648] Specific operation: Analyzes voice tone, speech content, and speech rate using an emotion analysis engine (e.g., IBM Watson Tone Analyzer) to recognize the user's emotions.

[1649] Input: Audio and text data

[1650] Output: Emotional state

[1651] Step 7:

[1652] The server reflects the emotion analysis results in the urgency assessment and determines the priority.

[1653] What you do: Reassess the urgency and prioritize it as high, medium, or low based on your emotional state and the request.

[1654] Input: Request, urgency, emotional state

[1655] Output: Priority

[1656] Step 8:

[1657] The server sends a notification to the nurse's terminal.

[1658] Specific behavior: The server generates notifications based on priority and sends them to the nurse's device. High priority notifications are sent immediately.

[1659] Input: Priorities, Needs, Emotional State

[1660] Output: Notification data

[1661] Step 9:

[1662] The user (nurse) receives the notification from the terminal and begins to respond.

[1663] Specific actions: A notification appears on the nurse's device, and an audio and visual alert is sent. The nurse checks the notification and begins to respond.

[1664] Input: Notification data

[1665] Output: Now supported

[1666] Step 10:

[1667] The terminal sends a timestamp of when the response started to the server and updates the response log.

[1668] Specific operation: The nurse presses the start response button on their device, the timestamp is sent to the server, and the log is updated.

[1669] Input: Start of response

[1670] Output: timestamp

[1671] Step 11:

[1672] When the user (nurse) has completed the treatment, the completion of the treatment is input into the terminal.

[1673] Specific action: The nurse completes the procedure and presses the "procedure completed" button on the terminal.

[1674] Input: Completed

[1675] Output: Completion signal

[1676] Step 12:

[1677] The device sends a timestamp of the completion to the server and updates the response history.

[1678] Specific operation: A completion timestamp is sent from the device to the server, and the action history is recorded as completed.

[1679] Input: Completion signal

[1680] Output: Completion timestamp and updated history

[1681] (Application example 2)

[1682] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1683] Conventional factory robot maintenance systems were sometimes slow to respond appropriately when an abnormality occurred, making it difficult to respond quickly to highly urgent events. Furthermore, there was a lack of a system that could automatically evaluate the nature and urgency of the abnormality and appropriately determine priorities. This resulted in an increased burden on maintenance staff and a decline in the overall factory's production efficiency, which became an issue.

[1684] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing voice data using generative artificial intelligence and evaluating the nature of the abnormality and the level of urgency, means for determining a priority based on the level of urgency, and means for sending a notification according to the priority to a staff member's terminal. This enables a prompt and appropriate response when an abnormality occurs in a robot in a factory, reducing the burden on maintenance staff and improving factory production efficiency.

[1685] A "user" is a person or device that accesses the system and reports an abnormality, and has the role of using a terminal to transmit identification information and voice data to the server.

[1686] "Terminal" means a device operated by a user to collect and transmit identification information and voice data, including computing devices such as smartphones and tablets.

[1687] "Identification information" is information for identifying a user or a robot, and is a unique identification code obtained by a radio frequency identification tag (RFID) or a barcode.

[1688] "Voice data" refers to voice information uttered by a user or a robot, and is data that is converted into text using voice recognition technology.

[1689] A "server" is a centralized computer system that receives data sent from devices, analyzes the data using generative artificial intelligence and emotion engines, and issues instructions for appropriate responses.

[1690] "Generative AI" is a type of AI technology that analyzes input voice data, converts it into text, and uses natural language processing technology to evaluate the content of requests and abnormalities.

[1691] "Natural language processing" is a technology that enables generative artificial intelligence to analyze text data, understand its meaning, and generate appropriate responses.

[1692] An "emotion engine" is a technology that analyzes the emotions of a user or situation from voice data or text data and reflects this in the assessment of urgency.

[1693] "Priority" is an evaluation criterion that classifies the priority of a response as "high," "medium," or "low" based on the results of analysis by the server.

[1694] "Notification" is information sent to staff terminals according to the priority determined by the server, and is received as an alert or message.

[1695] "Staff" refers to the people who receive notifications from the system and take appropriate action, including maintenance workers and nurses.

[1696] The present invention relates to a robot maintenance system used in a factory that enables a quick and appropriate response when an abnormality occurs. The present invention provides a system including the following main components:

[1697] 1. User abnormality notification

[1698] The user can use the terminal to report any abnormalities in the robot. When an abnormality occurs, the user can press a button on the terminal or report the abnormality by voice, which will trigger the system to notify the abnormality.

[1699] 2. Terminal data collection and transmission

[1700] The terminal collects the user's or robot's identification information and voice data and sends it to the server. The identification information is automatically read using a radio frequency identification tag (RFID) or barcode reader. The identification information and voice data acquired by the terminal are encrypted using a secure protocol (e.g., HTTPS) and sent to the server.

[1701] 3. Server Data Analysis

[1702] The server passes the received voice data to a generative artificial intelligence (AI) system, which converts it into text using speech recognition technology. The converted text data is then analyzed using natural language processing (NLP) technology to assess the nature of the anomaly and its urgency.

[1703] 4. Emotion analysis using an emotion engine

[1704] The server passes the analyzed text data to the emotion engine, which then reflects the emotion analysis in assessing the urgency of the anomaly. The emotion engine recognizes emotions from the voice tone, speech content, speech rate, or the robot's movement patterns, and adjusts the urgency accordingly.

[1705] 5. Prioritization

[1706] The server determines the priority of anomalies based on the analysis results of the generative artificial intelligence and emotion engine, and classifies the priority as "high," "medium," or "low."

[1707] 6. Notification to staff

[1708] The server sends notifications to the maintenance staff's terminals according to the determined priority. High-priority anomalies are notified immediately, while medium- and low-priority anomalies are notified at an appropriate time. The notification includes the nature of the anomaly, priority, and emotional state.

[1709] 7. Staff Response and Reporting

[1710] The user (maintenance staff) receives the notification from the terminal and begins to respond based on priority. When the staff member starts to respond, they input "response start" into the terminal. A timestamp of the response start is sent to the server and the response log is updated. When the response is completed, they input "response completion" into the terminal and a completion timestamp is sent to the server. The server receives the response completion timestamp, updates the response history, and records the abnormality processing status.

[1711] Specific examples

[1712] As example 1, we will introduce a scenario in which an emergency robot abnormality occurs.

[1713] Robot ID: robot_123

[1714] Error message: Critical failure, motor overheating.

[1715] Analyze the details of the error message with generative AI and use an emotion engine to assess urgency and display the emotional state.

[1716] The server analyzes this error message, rates it as high urgency, and sends an immediate notification to maintenance staff.

[1717] The above configuration enables prompt and appropriate response when an abnormality occurs in a robot in a factory, thereby reducing the burden on maintenance staff and improving factory production efficiency.

[1718] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1719] Step 1:

[1720] User abnormality notification begins

[1721] When an abnormality occurs in a robot in a factory, the user can report the abnormality by pressing a dedicated button on the terminal or by voice. The input is a voice message or button operation when an abnormality occurs. The output is a signal notifying the terminal of the abnormality. This causes the system to start receiving abnormality notifications.

[1722] Step 2:

[1723] Collection of identification information and voice data

[1724] The terminal automatically reads the user's or robot's identification information using an RFID or barcode reader, and also collects voice messages from the user as voice data. The input is the RFID tag or barcode and the voice message. The output is the identification information and voice data.

[1725] Step 3:

[1726] Sending data to the server

[1727] The device sends the collected identification information and voice data to the server using a secure protocol (e.g., HTTPS). The input is the identification information and voice data. The output is the data sent to the server.

[1728] Step 4:

[1729] Analysis of audio data

[1730] The server passes the received voice data to a generative AI, which converts the voice data into text using speech recognition technology. The input is the voice data, and the output is the converted text data. A generative AI model (e.g., GPT-4) is used for this process.

[1731] Step 5:

[1732] Text data analysis

[1733] The server analyzes the converted text data using a natural language processing (NLP) tool to evaluate the nature of the anomaly and its urgency. The input is the text data. The output is the analysis results, which include the nature of the anomaly and its urgency. An NLP tool (e.g., SpaCy) is used for this process.

[1734] Step 6:

[1735] Emotion Analysis

[1736] The server passes the analyzed text data to an emotion engine, which recognizes emotions from the voice tone and content of the speech. The input is text data. The output is the emotional state and urgency adjustment information as the result of emotion analysis. An emotion engine (e.g., IBM Watson Tone Analyzer) is used for this process.

[1737] Step 7:

[1738] Determining priorities

[1739] The server determines the priority of anomalies based on the analysis results of the generative artificial intelligence and emotion engine. The inputs are the details of the anomaly, the urgency, and the emotion analysis results. The output is the determined priority. Priorities are classified as "high," "medium," or "low."

[1740] Step 8:

[1741] Sending notifications to staff

[1742] The server sends a notification to the maintenance staff's terminal based on the determined priority. The input is the determined priority, the abnormality details, and the staff's emotional state. The output is a notification to the staff's terminal. The notification includes the abnormality details, priority, and emotional state.

[1743] Step 9:

[1744] Staff begins to respond

[1745] The user (maintenance staff) receives the notification and starts a response based on the priority. When the response starts, a timestamp of the response start is sent from the terminal to the server. The input is the notification and the start time, and the output is a response start log.

[1746] Step 10:

[1747] Response completion report

[1748] When the user (maintenance staff) completes the response, they input the response completion into the terminal and send a completion timestamp to the server. The input is the response completion time, and the output is a response completion log. The server receives the response completion timestamp, updates the response history, and records the abnormality processing status.

[1749] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1750] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1751] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1752] [Fourth embodiment]

[1753] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1754] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1755] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1756] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1757] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1758] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1759] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1760] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1761] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1762] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1763] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1764] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1765] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1766] This invention provides a method and system for incorporating generative artificial intelligence into a nurse call system to effectively manage patient contacts. This system works with the cooperation of users (patients), terminals, a server, and nurses.

[1767] System configuration

[1768] 1. User initiates nurse call

[1769] The user initiates contact by pressing the nurse call button on the terminal.

[1770] 2. Terminal data collection and transmission

[1771] The terminal obtains the user's identification information (eg, a radio frequency identification tag or a barcode) and collects the user's voice data.

[1772] The terminal transmits the acquired identification information and voice data to the server.

[1773] 3. Server Data Analysis

[1774] The server passes the received voice data to the generative artificial intelligence.

[1775] Generative AI uses speech recognition technology to convert voice data into text, and then uses natural language processing technology to analyze the text data, understand the content, and assess its urgency.

[1776] 4. Server Prioritization

[1777] The server evaluates the urgency of the nurse call and determines its priority based on the analysis results of the generative artificial intelligence.

[1778] 5. Sending notifications

[1779] The server sends notifications to the nurses' terminals based on the determined priority: high-priority requests are notified immediately, and low-priority requests are notified at an appropriate time.

[1780] 6. Nurse response and reporting

[1781] The user (nurse) receives the notification from the terminal and responds according to priority.

[1782] Nurses report their response status from their terminals to the server, and the progress and completion status of the response are updated.

[1783] Specific examples

[1784] Example 1: Emergency nurse call

[1785] User (patient) presses nurse call button:

[1786] Patient: "My stomach hurts, can you do something about it?"

[1787] The device sends the information to the server:

[1788] Patient ID: 12345

[1789] Audio data: "My stomach hurts, can you help me?"

[1790] The server analyzes the data using generative artificial intelligence:

[1791] Generative AI: Analyzes the content of voice data and determines "pain," "abdomen," and "urgency."

[1792] Urgency: High

[1793] The server decides the priority:

[1794] Priority: High

[1795] Immediate notification to nurses

[1796] The nurse begins to respond:

[1797] A nurse immediately rushes to the patient's room and provides treatment.

[1798] Sends a timestamp of when the response started to the server

[1799] Status report after completion of response:

[1800] The nurse completes the procedure and sends the end timestamp to the server.

[1801] The server logs the completion of the process and updates the emergency response history.

[1802] Example 2: Non-emergency nurse call

[1803] User (patient) presses nurse call button:

[1804] Patient: "What time is my next appointment?"

[1805] The device sends the information to the server:

[1806] Patient ID: 67890

[1807] Audio data: "What time is my next appointment?"

[1808] The server analyzes the data using generative artificial intelligence:

[1809] Generative artificial intelligence: Analyzes the content of voice data and determines "consultation time" and "information provision."

[1810] Urgency: Low

[1811] The server decides the priority:

[1812] Priority: Low

[1813] Providing timely notification to nurses

[1814] The nurse begins to respond:

[1815] The nurse will contact the patient after other emergency procedures have been completed.

[1816] Sends a timestamp of when the response started to the server

[1817] Status report after completion of response:

[1818] The nurse completes the information provision and sends the end timestamp to the server.

[1819] The server logs the completion of the process and updates the non-emergency history.

[1820] In this way, the system of the present invention is designed to improve nurse efficiency and increase patient satisfaction.

[1821] The processing flow will be explained below.

[1822] Step 1:

[1823] The user presses the nurse call button on the terminal, which sends out a signal to start a nurse call. Specifically, the terminal is waiting to receive user input and detects that the button has been pressed.

[1824] Step 2:

[1825] The terminal acquires the patient's identification information, which is automatically read using a radio frequency identification tag or barcode reader.

[1826] Step 3:

[1827] The user communicates their inquiry or request to the terminal by voice. For example, they can input a message such as "My stomach hurts."

[1828] Step 4:

[1829] The device collects the audio data and temporarily stores it as an audio file, which is accompanied by a timestamp and patient identification information.

[1830] Step 5:

[1831] The device sends the voice data and identification information to the server using the secure HTTPS protocol, and the data is encrypted.

[1832] Step 6:

[1833] The server passes the received voice data to the generative AI, which has a dedicated module for analyzing the voice data.

[1834] Step 7:

[1835] Generative artificial intelligence uses speech recognition technology to convert audio data into text, which is then passed on to the next processing stage for analysis.

[1836] Step 8:

[1837] Generative AI analyzes text data using natural language processing (NLP) techniques to assess the patient's needs and urgency, detecting keywords such as "pain," "fall," and "dyspnea."

[1838] Step 9:

[1839] Based on the results of the generative AI analysis, the server evaluates the urgency and determines the priority of the response, which is classified as "high," "medium," or "low."

[1840] Step 10:

[1841] The server sends notifications to the nurse's terminal according to priority. High-priority requests are notified immediately, while medium- and low-priority requests are notified at an appropriate time.

[1842] Step 11:

[1843] The terminal (nurse's device) receives notifications from the server and notifies the nurse with audio and visual alerts, including the request and priority.

[1844] Step 12:

[1845] The user (nurse) checks the notification and starts responding based on the specified priority. When the nurse starts responding, they input the start of response into the terminal.

[1846] Step 13:

[1847] The device sends a timestamp of when the response started to the server, and the server confirms that the response started and records it in a log.

[1848] Step 14:

[1849] The user (nurse) visits the patient and responds. Depending on the response, necessary measures are taken.

[1850] Step 15:

[1851] When the user (nurse) completes the treatment, they input the completion of the treatment into the terminal. The terminal sends a timestamp of the completion of the treatment to the server.

[1852] Step 16:

[1853] The server receives the timestamp of the response completion and updates the response history, thereby recording the nurse call processing status as completed in the log.

[1854] This processing flow allows the nurse call system to efficiently process patient requests, reducing the burden on nurses and improving patient satisfaction.

[1855] Example 1

[1856] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1857] In conventional nurse call systems, nurses may not be able to respond appropriately to calls from patients with different levels of urgency. This makes it difficult to respond quickly to calls with high urgency, which can lead to a decrease in patient safety and satisfaction. Furthermore, nurses may not be able to perform their work efficiently, which can result in excessive burdens. Therefore, the present invention aims to provide a system that effectively manages calls from patients, reducing the workload of nurses while enabling them to respond quickly and accurately.

[1858] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1859] In this invention, the server includes a means for a user to initiate a nurse call on a terminal, a means for the terminal to send the user's identification information and voice data to the server, a means for the server to analyze the voice data using generative artificial intelligence and evaluate the request content and urgency, a means for the server to determine the priority of the nurse call based on the analysis results, a means for the server to send a notification according to the urgency to the nurse's terminal, a means for the nurse to report the response status to the server via the terminal, and a means for the server to record the completion of processing in a log and update the response history. This enables a quick and accurate response that takes the urgency into consideration and reduces the workload of nurses.

[1860] "User" refers to the person who operates the terminal that initiates the nurse call.

[1861] "Terminal" refers to a device that acquires user identification information and voice data and transmits them to a server.

[1862] "Identification information" is unique information for identifying a user, and is obtained using, for example, a radio frequency identification tag or a barcode.

[1863] "Voice data" refers to data including what the user said when calling the nurse.

[1864] "Server" refers to a device or system that receives data sent from a terminal and analyzes it using generative artificial intelligence.

[1865] "Generative artificial intelligence" refers to an artificial intelligence model that uses voice recognition technology and natural language processing technology to analyze voice data.

[1866] "Voice recognition technology" refers to technology that converts voice data into text data.

[1867] "Natural language processing technology" refers to technology for analyzing text data and understanding its content.

[1868] "Urgency" refers to an indicator that evaluates how urgent the content of a nurse call is.

[1869] "Priority" refers to the criteria for determining the order of response based on the urgency of the nurse call.

[1870] "Notification" refers to a message that sends information about the nurse call and the level of urgency to the nurse's terminal.

[1871] "Caregiver" refers to a medical professional who takes action based on the notification sent.

[1872] "Response status" refers to the progress of a nurse's response to a nurse call.

[1873] "Log" refers to data that a server uses to record the completion of processing and response history.

[1874] This invention provides a method and system for incorporating generative artificial intelligence into a nurse call system to effectively manage calls from patients. This system works through the cooperation of users (patients), terminals, a server, and caregivers (nurses).

[1875] The system is implemented using the following hardware and software.

[1876] Terminal: A device that acquires a user's identification information and voice data and transmits them to a server. Specifically, a smart device equipped with a radio frequency identification tag reader or barcode reader is used.

[1877] Server: A device that uses generative artificial intelligence to process and analyze data received from the terminal, and sends notifications to the caregiver's terminal based on the determined priority. Specific implementation examples include cloud servers and on-premise servers.

[1878] Generative artificial intelligence: Software that analyzes audio data using speech recognition technologies (e.g., Google Cloud Speech-to-Text) and natural language processing technologies (e.g., Google Cloud Natural Language API).

[1879] This system allows users (patients) to quickly call a nurse in an emergency, and nurses can respond efficiently by receiving notifications according to priority.

[1880] The specific operation flow is shown below.

[1881] 1. User (patient) initiates nurse call

[1882] The user presses the nurse call button on the terminal to initiate a call. For example, a patient presses the nurse call button when they feel a stomach ache.

[1883] 2. Terminal data collection and transmission

[1884] The terminal acquires the user's identification information (e.g., radio frequency identification tag or barcode) and collects the user's voice data. For example, it reads "Patient ID: 12345" and sends the voice data "My stomach hurts. Can you do something about it?" to the server.

[1885] 3. Server Data Analysis

[1886] The server passes the received voice data to a generative AI. The generative AI converts the voice data into text using speech recognition technology, and then analyzes the text data using natural language processing technology to understand the content and assess the urgency. For example, the voice data "My stomach hurts. Can you do something?" would be analyzed as "pain," "abdomen," and "high urgency."

[1887] 4. Server Prioritization

[1888] The server evaluates the urgency of the nurse call based on the analysis results of the generative AI and determines the priority. For example, if the urgency is evaluated as "high," the priority will be set to "high."

[1889] 5. Sending notifications

[1890] The server sends a notification to the nurse's terminal based on the determined priority. For example, if the priority is "High," an immediate notification is sent. The notification content may include something like, "Please go immediately to the room of patient ID: 12345. The patient is complaining of stomach pain."

[1891] 6. Caregiver response and reporting

[1892] The user (caregiver) receives the notification from the terminal and responds according to priority. For example, the nurse who received the notification rushes to the patient's room. The caregiver reports the response status from the terminal to the server, records the completion of the process in a log, and updates the response history. For example, timestamps at the start and end of the response are sent to the server.

[1893] Specific examples

[1894] Example 1: Emergency nurse call

[1895] User (patient) presses nurse call button:

[1896] Patient: "My stomach hurts, can you do something about it?"

[1897] The device sends the information to the server:

[1898] Patient ID: 12345

[1899] Audio data: "My stomach hurts, can you help me?"

[1900] The server analyzes the data using generative artificial intelligence:

[1901] Generative AI: Analyzes the content of the voice data and determines whether the condition is "painful," "abdominal," or "high urgency."

[1902] Urgency: High

[1903] The server decides the priority:

[1904] Priority: High

[1905] Immediate notification to caregivers

[1906] Caregiver initiates response:

[1907] A caregiver immediately goes to the patient's room and provides treatment.

[1908] Sends a timestamp of when the response started to the server

[1909] Status report after completion of response:

[1910] The caregiver completes the procedure and sends the completion timestamp to the server.

[1911] The server logs the completion of the process and updates the emergency response history.

[1912] Example 2: Non-emergency nurse call

[1913] User (patient) presses nurse call button:

[1914] Patient: "What time is my next appointment?"

[1915] The device sends the information to the server:

[1916] Patient ID: 67890

[1917] Audio data: "What time is my next appointment?"

[1918] The server analyzes the data using generative artificial intelligence:

[1919] Generative artificial intelligence: Analyzes the content of voice data and determines "consultation time" and "information provision."

[1920] Urgency: Low

[1921] The server decides the priority:

[1922] Priority: Low

[1923] Providing timely notification to caregivers

[1924] Caregiver initiates response:

[1925] After other emergency procedures are completed, the caregiver contacts the patient.

[1926] Sends a timestamp of when the response started to the server

[1927] Status report after completion of response:

[1928] The caregiver completes the information submission and sends an end timestamp to the server.

[1929] The server logs the completion of the process and updates the non-emergency response history.

[1930] In this manner, the system of the present invention is designed to improve caregiver efficiency and increase patient satisfaction.

[1931] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1932] Step 1:

[1933] User (patient) initiates nurse call

[1934] This step begins when the user presses the nurse call button on the terminal. Specifically, the patient presses the button when they feel a stomach ache.

[1935] Input: User (patient) presses nurse call button

[1936] Output: Nurse call start signal

[1937] Step 2:

[1938] Terminal data collection and transmission

[1939] Upon receiving the nurse call start signal, the terminal simultaneously acquires the user's identification information (e.g., radio frequency identification tag or barcode) and collects the user's voice data. This data is then sent to the server. Specifically, the terminal reads "Patient ID: 12345" and sends the voice data "My stomach hurts. Can you do something about this?" to the server.

[1940] Input: Nurse call start signal, user identification information, voice data

[1941] Output: Sending a data packet containing the identification information and audio data to the server

[1942] Step 3:

[1943] Receiving data from the server and preparing for analysis

[1944] The server receives the data packets sent from the terminal and prepares them for analysis. In this step, preprocessing is performed to pass the received data to the generative artificial intelligence.

[1945] Input: Data packet containing identification information and voice data

[1946] Output: Data ready for analysis

[1947] Step 4:

[1948] Data analysis using generative artificial intelligence

[1949] The server passes the prepared data to the generative AI, which analyzes the voice data. The generative AI uses speech recognition technology to convert the voice data into text data, and then uses natural language processing technology to analyze the text data and evaluate the request's content and urgency. For example, it analyzes the voice data "My stomach hurts. Can you do something about this?" and determines that the request is "pain," "abdomen," and "high urgency."

[1950] Input: Data ready for analysis

[1951] Output: Text data and a severity rating based on that text

[1952] Step 5:

[1953] Server Prioritization

[1954] The server evaluates the urgency of the nurse call based on the analysis results of the generative artificial intelligence and determines the priority based on that evaluation. For example, if the urgency is determined to be "high," the priority is set to "high."

[1955] Input: Text data and urgency assessment using generative AI

[1956] Output: Priority determination

[1957] Step 6:

[1958] Sending notifications

[1959] The server then sends a notification to the nurse's terminal based on the determined priority. For example, if the priority is "High," it will immediately send a notification stating, "Please go immediately to the room of patient ID: 12345. The patient is complaining of stomach pain."

[1960] Input: Priority determination

[1961] Output: Notification to caregiver's terminal

[1962] Step 7:

[1963] Caregiver response and reporting

[1964] The user (caregiver) receives the notification from the terminal and responds according to the priority. For example, the caregiver who receives the notification rushes to the patient's room.

[1965] Input: Notification to caregiver's terminal

[1966] Output: Action to start the response

[1967] The nurse then reports the treatment status from the terminal to the server. The progress and completion status of the treatment are recorded using timestamps. Specifically, at the start and end of the treatment, the nurse presses the "Start Treatment" and "Complete Treatment" buttons to send timestamps to the server.

[1968] Input: Response start and completion timestamps

[1969] Output: Updates the corresponding log to the server

[1970] This enables the entire system to respond quickly and appropriately, taking into account the level of urgency, reducing the workload of caregivers and increasing patient satisfaction.

[1971] (Application example 1)

[1972] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1973] Existing security systems often fail to properly detect and respond to suspicious activities and emergencies, increasing the burden on security guards. In particular, systems lack mechanisms for proper prioritization and immediate notification in situations requiring rapid response based on urgency. Furthermore, even in situations where rapid backup is required in an emergency, the time lag in manual communication is a problem.

[1974] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1975] In this invention, the server includes means for acquiring audio or video data in real time and detecting suspicious activity or emergency situations using generative artificial intelligence, means for automatically making additional backup requests in the event of an emergency, and means for analyzing the acquired audio data and evaluating the request content and urgency, thereby enabling rapid detection and response to suspicious activity or emergency situations, and rapid backup requests in the event of an emergency.

[1976] A "user" is a human entity that uses a terminal to initiate a call.

[1977] A "terminal" is an electronic device that acquires voice data and identification information and transmits them to a server.

[1978] "Identification information" refers to information that identifies a particular user or patient, and includes radio frequency identification tags and bar codes.

[1979] A "server" is a computer system that processes and manages data on a network.

[1980] "Generative AI" is an AI technology that analyzes voice and text data to understand the content and assess its urgency.

[1981] "Voice data" is data containing voice information uttered by a user.

[1982] "Urgency" is a measure of the importance and urgency of a request or event to be handled.

[1983] "Priority" refers to the order in which events or requests to be addressed are ranked according to their importance.

[1984] "Notification" is the act of informing a terminal of a specific situation by sending information from a server to the terminal.

[1985] "Response status" is information that indicates the progress and results of actions taken in response to requests or events.

[1986] "Audio or video data" refers to data that includes audio and video information captured in real time.

[1987] "Suspicious activity" is any suspicious behavior, including fraudulent or unusual actions or movements.

[1988] An "emergency" is an urgent situation requiring immediate action.

[1989] A "backup request" is a request for additional assistance or response in an emergency.

[1990] The system for realizing this invention is mainly composed of three elements: a user, a terminal, and a server. A specific embodiment of this system will be described below.

[1991] The basic configuration of the system is that the user initiates a call using their device, which then sends identification information and voice data to the server. The server uses generative artificial intelligence to analyze the voice data and evaluate the request content and urgency. Based on the analysis results, the server determines a priority according to the urgency and sends a notification to the responder's device. The responder receives the notification and reports the response status to the server, improving overall work efficiency.

[1992] The server also captures audio and video data in real time and uses generative artificial intelligence to detect suspicious activity or emergencies. If suspicious activity or an emergency is detected, the server immediately assesses the severity and automatically requests additional backups if necessary.

[1993] Hardware and Software Use

[1994] 1. Device:

[1995] Wearable devices such as smartphones, smart glasses, and head-mounted displays will be used.

[1996] It is equipped with a microphone and a camera for capturing audio and video data.

[1997] 2. Software:

[1998] Speech Recognition: Use the SpeechRecognition library to convert voice data into text.

[1999] Sending data: Use the requests library to send data from the terminal to the server.

[2000] Generative AI: Using artificial intelligence models, such as GPT-3, to perform advanced analysis.

[2001] Specific examples

[2002] For example, imagine a suspicious individual appears while a security guard at a shopping mall is patrolling the area wearing smart glasses. If the suspicious individual utters a clear indication of fraudulent activity, such as "I'm going to break the camera," the microphone in the smart glasses captures the audio and immediately sends it to the server. The server then uses generative artificial intelligence to analyze the audio and evaluate whether or not there is fraudulent activity and the urgency of the situation. It then determines a priority and sends an immediate notification to the security guard's device. The guard who receives this notification can respond quickly and report the response status to the server in real time, thereby improving the overall security level.

[2003] Prompt Sentence Examples

[2004] Dear customer, please tell us more about your current situation. Is it an emergency? Do you need assistance?

[2005] Based on these specific examples, this system can realize a swift and appropriate response according to the level of urgency, which will significantly improve the security of facilities and locations, enabling effective measures to be taken in situations where a rapid response is required.

[2006] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2007] Step 1:

[2008] A user initiates a call on a terminal. The input is the user pressing the call button, and the terminal starts capturing audio. The output is the captured audio data and the user's identification information.

[2009] Step 2:

[2010] The terminal sends the user's identification information and voice data to the server. The input is the voice data and identification information acquired by the terminal, which are then sent to the server via the network. The output is the voice data and identification information passed to the server.

[2011] Step 3:

[2012] The server analyzes the voice data using a generative artificial intelligence model. The voice data is sent as input, and the generative artificial intelligence is used to convert the voice into text and analyze the content. Specifically, the voice data is converted into text data using voice recognition technology, and then the content of the text is understood using natural language processing technology, and the request content and urgency are evaluated. The analysis results are generated as output.

[2013] Step 4:

[2014] Based on the analysis results, the server evaluates the request content and urgency and determines the priority. The input is the analysis results from the generative AI, and based on this, the urgency is evaluated and priorities are set. For example, an evaluation such as "high urgency" or "low urgency" is made. A priority list is generated as the output.

[2015] Step 5:

[2016] The server sends notifications to the responder's device according to the priority. The input is a priority list, and notifications are sent to the responder's device based on this information. If the priority is high, immediate notification is sent, and if the priority is low, timely notification is sent. The output is a notification that arrives on the responder's device.

[2017] Step 6:

[2018] The responder reports the response status to the server via the terminal. The input includes details of the response performed by the responder and the progress status, which are reported via the terminal. Specifically, the timestamps for when the response started and when it was completed are reported. The output is a record of the response status on the server.

[2019] Step 7:

[2020] The server acquires audio or video data in real time and uses generative artificial intelligence to detect suspicious activity or emergencies. Real-time audio or video data is input, which is then analyzed by generative artificial intelligence. Specifically, voice recognition and video analysis are performed to detect suspicious sounds and behavior. The output is a detection result of suspicious activity or an emergency.

[2021] Step 8:

[2022] In an emergency, the server automatically requests additional backups. The input is the detection result of suspicious activity or an emergency, and the backup request is made according to the level of urgency. Specifically, if the level of urgency is high, a request for assistance is automatically sent to additional responders. The output is a backup request sent to the responder.

[2023] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2024] This invention relates to a system that incorporates generative artificial intelligence and an emotion engine into a nurse call system to accurately manage patient contact and support nurses' efficient responses. This system consists of five main components: the user (patient), terminal, server, nurse, and emotion engine.

[2025] System configuration

[2026] 1. User initiates nurse call

[2027] The user initiates contact by pressing the nurse call button on the terminal. This action causes the system to start the nurse call reception process.

[2028] 2. Terminal data collection and transmission

[2029] The terminal obtains the user's identification information and collects the voice data, which is automatically read using a radio frequency identification tag or bar code reader.

[2030] The device sends the acquired identification information and voice data to the server. This data is encrypted using a secure protocol (e.g., HTTPS) and then transmitted.

[2031] 3. Server Data Analysis

[2032] The server passes the received voice data to the generative AI, which converts the voice data into text using voice recognition technology.

[2033] The server receives the converted text data, analyzes it using natural language processing (NLP) technology, and evaluates the content and urgency of the request.

[2034] 4. Emotion analysis using an emotion engine

[2035] The server passes the acquired voice and text data to the emotion engine, which analyzes the voice tone, speech content, and speech rate to recognize the user's emotions.

[2036] The server reflects the recognized emotion in the assessment of urgency and adjusts the priority as necessary.

[2037] 5. Server Prioritization

[2038] The server determines the priority of nurse call requests based on the analysis results of the generative artificial intelligence and emotion engine. Priority is classified as "high," "medium," or "low."

[2039] 6. Sending Notifications

[2040] The server sends notifications to the nurses' terminals based on the determined priority. High-priority requests are notified immediately, and medium- and low-priority requests are notified at an appropriate time.

[2041] The user (nurse) receives notifications from the device and confirms them with audio and visual alerts, which include the request, priority, and the user's emotional state.

[2042] 7. Nurse response and reporting

[2043] The user (nurse) checks the notification on the terminal and starts responding based on the priority. When the nurse starts responding, they input the start of response on the terminal.

[2044] The terminal sends a timestamp of when the response started to the server, and the response log is updated.

[2045] When the user (nurse) completes the treatment, they input the completion of the treatment into the terminal. The terminal records a timestamp of the completion of the treatment and sends it to the server.

[2046] The server receives the timestamp of the response completion and updates the response history, thereby recording the nurse call processing status as completed.

[2047] Specific examples

[2048] Example 1: Emergency nurse call

[2049] User (patient) presses nurse call button:

[2050] Patient: "My stomach hurts, can you do something about it?"

[2051] The device sends the information to the server:

[2052] Patient ID: 12345

[2053] Audio data: "My stomach hurts, can you help me?"

[2054] The server analyzes the data using generative artificial intelligence:

[2055] Generative AI: Analyzes the content of voice data and determines "pain," "abdomen," and "urgency."

[2056] Urgency: High

[2057] The server performs sentiment analysis using the emotion engine:

[2058] Emotion Engine: Analyzes voice tone and speech rate to detect the user's anxiety.

[2059] Detecting anxiety will set the level of urgency even higher.

[2060] The server decides the priority:

[2061] Priority: High

[2062] Immediate notification to nurses

[2063] The nurse begins to respond:

[2064] A nurse immediately rushes to the patient's room and provides treatment.

[2065] Sends a timestamp of when the response started to the server

[2066] Status report after completion of response:

[2067] The nurse completes the procedure and sends the end timestamp to the server.

[2068] The server logs the completion of the process and updates the emergency response history.

[2069] Example 2: Non-emergency nurse call

[2070] User (patient) presses nurse call button:

[2071] Patient: "What time is my next appointment?"

[2072] The device sends the information to the server:

[2073] Patient ID: 67890

[2074] Audio data: "What time is my next appointment?"

[2075] The server analyzes the data using generative artificial intelligence:

[2076] Generative artificial intelligence: Analyzes the content of voice data and determines "consultation time" and "information provision."

[2077] Urgency: Low

[2078] The server performs sentiment analysis using the emotion engine:

[2079] Emotion engine: Analyzes speech content and tone to determine whether the user is calm.

[2080] The server decides the priority:

[2081] Priority: Low

[2082] Providing timely notification to nurses

[2083] The nurse begins to respond:

[2084] The nurse will contact the patient after other emergency procedures have been completed.

[2085] Sends a timestamp of when the response started to the server

[2086] Status report after completion of response:

[2087] The nurse completes the information provision and sends the end timestamp to the server.

[2088] The server logs the completion of the process and updates the non-emergency history.

[2089] This system will improve the work efficiency of nurses, enable them to respond in a way that takes patients' feelings into consideration, and increase patient satisfaction.

[2090] The processing flow will be explained below.

[2091] Step 1:

[2092] The user presses the nurse call button on the terminal. This sends a signal to start the nurse call. Specifically, the terminal waits for user input and detects that the button has been pressed.

[2093] Step 2:

[2094] The terminal acquires the user's identification information. The identification information is automatically read using, for example, a radio frequency identification tag or a barcode reader. This allows identification information such as a patient ID to be acquired.

[2095] Step 3:

[2096] The user speaks their inquiry or request to the terminal. For example, they can input messages such as "I have a stomachache" or "Please tell me when my next appointment is."

[2097] Step 4:

[2098] The device collects the audio data and temporarily stores it as an audio file, which is accompanied by a timestamp and patient identification information.

[2099] Step 5:

[2100] The device sends the voice data and identification information to the server. The data is encrypted and sent securely using the HTTPS protocol.

[2101] Step 6:

[2102] The server passes the received voice data and identification information to the generative AI, which then converts the voice data into text using voice recognition technology.

[2103] Step 7:

[2104] Generative artificial intelligence uses speech recognition technology to convert voice data into text, and then passes the converted text data to the next analysis stage.

[2105] Step 8:

[2106] Generative AI analyzes text data using natural language processing (NLP) techniques to assess the patient's needs and urgency, detecting keywords such as "pain," "fall," and "dyspnea."

[2107] Step 9:

[2108] The server passes the acquired text data to the emotion engine, which analyzes the voice tone, speech content, and speech rate to recognize the user's emotions.

[2109] Step 10:

[2110] An emotion engine analyzes voice tone and speaking rate to assess the user's emotion (e.g., anxious, calm, nervous, etc.).

[2111] Step 11:

[2112] The server integrates data from the emotion engine and generative artificial intelligence, and determines priorities based on the request content, urgency, and the user's emotional evaluation. For example, if anxiety is high, the urgency level is set high.

[2113] Step 12:

[2114] The server sends notifications to the nurse's terminal based on priority: high-priority requests are notified immediately, and medium- and low-priority requests are notified at an appropriate time.

[2115] Step 13:

[2116] The terminal (nurse's device) receives notifications from the server and notifies the nurse with audio and visual alerts, including the request, priority, and the user's emotional state.

[2117] Step 14:

[2118] The user (nurse) checks the notification and starts responding based on the specified priority. When the nurse starts responding, they input the start of response into the terminal.

[2119] Step 15:

[2120] The device sends a timestamp of when the response started to the server, and the server confirms that the response started and records it in a log.

[2121] Step 16:

[2122] The user (nurse) visits the patient and provides treatment. Specifically, the user checks the patient's symptoms and provides the necessary treatment.

[2123] Step 17:

[2124] When the user (nurse) completes the treatment, they input the completion of the treatment into the terminal. The terminal records a timestamp of the completion of the treatment and sends it to the server.

[2125] Step 18:

[2126] The server receives the timestamp of the response completion and updates the response history, thereby recording the nurse call processing status as completed.

[2127] With this processing flow, the nurse call system of the present invention not only efficiently processes patient requests and reduces the burden on nurses, but also enables prompt and appropriate responses that take into consideration the patient's feelings.

[2128] Example 2

[2129] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2130] Conventional nurse call systems have difficulty accurately grasping the level of urgency and emotional state of patients, resulting in delays in nurses' responses and an inability to assign appropriate priorities. As a result, patient satisfaction has decreased and nurses' workload has increased. To solve this, a system that can respond quickly and accurately while also taking into account the patient's emotional state is needed.

[2131] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2132] In this invention, the server includes means for analyzing voice data using generative artificial intelligence and evaluating the content and urgency of requests, means for analyzing the user's emotions using an emotion analysis engine and reflecting this in the urgency evaluation, and means for determining priorities based on the urgency. This enables responses that take into account not only the content of the patient's request but also their emotional state, supporting nurses in responding quickly and appropriately, improving patient satisfaction and reducing the workload of nurses.

[2133] A "user" is a patient or a patient's representative who uses the system to make a nurse call.

[2134] A "terminal" is a device that allows a user to initiate a nurse call, and is a device that has the role of collecting patient identification information and voice data and transmitting them to a server.

[2135] The "server" is a central processing unit that receives data sent from the terminal, analyzes it using generative artificial intelligence and an emotion analysis engine, and sends appropriate notifications to nurses.

[2136] "Generative AI" is an AI technology that analyzes voice data, converts it into text, and uses natural language processing technology to evaluate the content and urgency of requests.

[2137] An "emotion analysis engine" is a technology that analyzes the tone of voice, speech content, and speech rate to recognize the user's emotional state.

[2138] "Urgency" is an evaluation index that indicates the urgency of responding to a nurse call from a user, and is an important parameter for determining the priority of nurses' responses.

[2139] "Priority" is an index that indicates the priority of the response that the server determines for a nurse call, and indicates the order in which requests are responded to.

[2140] A "nurse" is a medical professional who is responsible for responding to nurse calls from users, receiving notifications via a terminal, and taking appropriate action.

[2141] A "notification" is a message sent from the server to the nurse's terminal, with a priority set based on the degree of urgency, to prompt the nurse to take action.

[2142] This invention incorporates generative artificial intelligence and an emotion analysis engine into a nurse call system to manage patient calls with high accuracy and support nurses' efficient responses. This system consists of five main components: the user, the terminal, the server, the nurse, and the emotion analysis engine. The specific functions of each component and the operation of the entire system are explained below.

[2143] User initiates nurse call

[2144] The user initiates contact using the nurse call button. This action initiates the nurse call reception process. The user can be the patient or their representative.

[2145] Terminal data collection and transmission

[2146] When the nurse call button is pressed, the terminal collects the patient's identification information and voice data. The identification information is automatically read using a radio frequency identification tag reader or barcode reader. The terminal then transmits this information to a server. This data transmission is encrypted using a secure protocol (e.g., HTTPS).

[2147] Server data analysis

[2148] The server receives the voice data sent from the device and passes it to a generative AI. The generative AI converts the voice data into text using speech recognition technology (e.g., Google Cloud Speech-to-Text). The server then analyzes the converted text using natural language processing technology (e.g., spaCy, BERT) to evaluate the content and urgency of the request.

[2149] Emotion analysis using an emotion analysis engine

[2150] The server passes the voice and text data to an emotion analysis engine (e.g., IBM Watson Tone Analyzer), which analyzes the voice tone, speech content, and speech rate to recognize the user's emotions. This allows the emotion analysis results to be reflected in the urgency assessment. For example, if the user feels highly anxious, the urgency can be set even higher.

[2151] Prioritizing and sending notifications

[2152] The server determines the priority of the nurse call based on the analysis results and sends notifications to the nurse's device accordingly. High-priority notifications are sent immediately, while medium- and low-priority notifications are sent at an appropriate time. Notifications include the request content, priority, and the user's emotional state.

[2153] Nurse's response and report

[2154] The user (nurse) checks the notification from the terminal and begins responding based on the priority. When the nurse starts responding, they input the start of response into the terminal, and the timestamp is sent to the server. When the response ends, the end timestamp is also sent from the terminal to the server. This causes the server to update the response history and complete the entire process.

[2155] Specific examples

[2156] Example 1: Emergency nurse call

[2157] User (patient) presses nurse call button:

[2158] Patient: "My stomach hurts, can you do something about it?"

[2159] The device sends the information to the server:

[2160] Patient ID: 12345

[2161] Audio data: "My stomach hurts, can you help me?"

[2162] The server analyzes the data using generative artificial intelligence:

[2163] Generative artificial intelligence converts voice data into text and analyzes the request using natural language processing technology.

[2164] Urgency: High

[2165] The server performs sentiment analysis using the sentiment analysis engine:

[2166] The sentiment analysis engine detects the user's anxiety and further increases the level of urgency.

[2167] The server decides the priority:

[2168] Priority: High

[2169] Immediate notification to nurses

[2170] The nurse begins:

[2171] A nurse immediately rushes to the patient's room and provides treatment.

[2172] Sends a timestamp of when the response started to the server

[2173] Status report after completion of response:

[2174] The nurse completes the procedure and sends the completion timestamp to the server.

[2175] The server logs the completion of the process and updates the emergency response history.

[2176] Example 2: Non-emergency nurse call

[2177] User (patient) presses nurse call button:

[2178] Patient: "What time is my next appointment?"

[2179] The device sends the information to the server:

[2180] Patient ID: 67890

[2181] Audio data: "What time is my next appointment?"

[2182] The server analyzes the data using generative artificial intelligence:

[2183] Generative artificial intelligence converts voice data into text and analyzes the request using natural language processing technology.

[2184] Urgency: Low

[2185] The server performs sentiment analysis using the sentiment analysis engine:

[2186] The sentiment analysis engine determines that the user is calm.

[2187] The server decides the priority:

[2188] Priority: Low

[2189] Providing timely notification to nurses

[2190] The nurse begins:

[2191] The nurse will contact the patient after other emergency procedures have been completed.

[2192] Sends a timestamp of when the response started to the server

[2193] Status report after completion of response:

[2194] The nurse completes the information provision and sends the end timestamp to the server.

[2195] The server logs the completion of the process and updates the non-emergency history.

[2196] By utilizing a generative AI model and an emotion analysis engine, this system enables quick and appropriate responses that take into account the patient's emotions, thereby reducing the burden on nurses and improving patient satisfaction.

[2197] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2198] Step 1:

[2199] The user presses the nurse call button.

[2200] Specific operation: The patient initiates contact by pressing the nurse call button next to the bed, which triggers the terminal to start accepting and processing the nurse call.

[2201] Input: User action (pressing nurse call button)

[2202] Output: Nurse call start signal

[2203] Step 2:

[2204] The terminal collects patient identification information and voice data.

[2205] What it does: The device's microphone records the audio and uses a radio frequency identification tag reader or barcode reader to read the patient's identification information.

[2206] Input: Nurse call start signal

[2207] Output: Audio data and identification information

[2208] Step 3:

[2209] The terminal transmits the collected data to the server.

[2210] Specific operation: The device encrypts the collected voice data and patient identification information using the HTTPS protocol and sends it to the server.

[2211] Input: Audio data and identification information

[2212] Output: Encrypted data packet

[2213] Step 4:

[2214] The server receives the voice data and passes it to the generative artificial intelligence.

[2215] Specific operation: Voice data and identification information are stored in a database, and the voice data is sent to generative artificial intelligence for voice recognition.

[2216] Input: Encrypted data packet

[2217] Output: Text data (speech recognition results)

[2218] Step 5:

[2219] The server analyzes the text data using natural language processing technology and evaluates the request content and urgency.

[2220] What it does: The server uses a natural language processing library (e.g., spaCy) to analyze the text data and extract the necessary information (e.g., "pain," "abdomen," "urgent," etc.).

[2221] Input: Text data

[2222] Output: Request details and urgency

[2223] Step 6:

[2224] The server passes the voice and text data to an emotion analysis engine to analyze the user's emotions.

[2225] Specific operation: Analyzes voice tone, speech content, and speech rate using an emotion analysis engine (e.g., IBM Watson Tone Analyzer) to recognize the user's emotions.

[2226] Input: Audio and text data

[2227] Output: Emotional state

[2228] Step 7:

[2229] The server reflects the emotion analysis results in the urgency assessment and determines the priority.

[2230] What you do: Reassess the urgency and prioritize it as high, medium, or low based on your emotional state and the request.

[2231] Input: Request, urgency, emotional state

[2232] Output: Priority

[2233] Step 8:

[2234] The server sends a notification to the nurse's terminal.

[2235] Specific behavior: The server generates notifications based on priority and sends them to the nurse's device. High priority notifications are sent immediately.

[2236] Input: Priorities, Needs, Emotional State

[2237] Output: Notification data

[2238] Step 9:

[2239] The user (nurse) receives the notification from the terminal and begins to respond.

[2240] Specific actions: A notification appears on the nurse's device, and an audio and visual alert is sent. The nurse checks the notification and begins to respond.

[2241] Input: Notification data

[2242] Output: Now supported

[2243] Step 10:

[2244] The terminal sends a timestamp of when the response started to the server and updates the response log.

[2245] Specific operation: The nurse presses the start response button on their device, the timestamp is sent to the server, and the log is updated.

[2246] Input: Start of response

[2247] Output: timestamp

[2248] Step 11:

[2249] When the user (nurse) has completed the treatment, the completion of the treatment is input into the terminal.

[2250] Specific action: The nurse completes the procedure and presses the "procedure completed" button on the terminal.

[2251] Input: Completed

[2252] Output: Completion signal

[2253] Step 12:

[2254] The device sends a timestamp of the completion to the server and updates the response history.

[2255] Specific operation: A completion timestamp is sent from the device to the server, and the action history is recorded as completed.

[2256] Input: Completion signal

[2257] Output: Completion timestamp and updated history

[2258] (Application example 2)

[2259] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2260] Conventional factory robot maintenance systems were sometimes slow to respond appropriately when an abnormality occurred, making it difficult to respond quickly to highly urgent events. Furthermore, there was a lack of a system that could automatically evaluate the nature and urgency of the abnormality and appropriately determine priorities. This resulted in an increased burden on maintenance staff and a decline in the overall factory's production efficiency, which became an issue.

[2261] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for analyzing voice data using generative artificial intelligence and evaluating the nature of the abnormality and the level of urgency, means for determining a priority based on the level of urgency, and means for sending a notification according to the priority to a staff member's terminal. This enables a prompt and appropriate response when an abnormality occurs in a robot in a factory, reducing the burden on maintenance staff and improving factory production efficiency.

[2262] A "user" is a person or device that accesses the system and reports an abnormality, and has the role of using a terminal to transmit identification information and voice data to the server.

[2263] "Terminal" means a device operated by a user to collect and transmit identification information and voice data, including computing devices such as smartphones and tablets.

[2264] "Identification information" is information for identifying a user or a robot, and is a unique identification code obtained by a radio frequency identification tag (RFID) or a barcode.

[2265] "Voice data" refers to voice information uttered by a user or a robot, and is data that is converted into text using voice recognition technology.

[2266] A "server" is a centralized computer system that receives data sent from devices, analyzes the data using generative artificial intelligence and emotion engines, and issues instructions for appropriate responses.

[2267] "Generative AI" is a type of AI technology that analyzes input voice data, converts it into text, and uses natural language processing technology to evaluate the content of requests and abnormalities.

[2268] "Natural language processing" is a technology that enables generative artificial intelligence to analyze text data, understand its meaning, and generate appropriate responses.

[2269] An "emotion engine" is a technology that analyzes the emotions of a user or situation from voice data or text data and reflects this in the assessment of urgency.

[2270] "Priority" is an evaluation criterion that classifies the priority of a response as "high," "medium," or "low" based on the results of analysis by the server.

[2271] "Notification" is information sent to staff terminals according to the priority determined by the server, and is received as an alert or message.

[2272] "Staff" refers to the people who receive notifications from the system and take appropriate action, including maintenance workers and nurses.

[2273] The present invention relates to a robot maintenance system used in a factory that enables a quick and appropriate response when an abnormality occurs. The present invention provides a system including the following main components:

[2274] 1. User abnormality notification

[2275] The user can use the terminal to report any abnormalities in the robot. When an abnormality occurs, the user can press a button on the terminal or report the abnormality by voice, which will trigger the system to notify the abnormality.

[2276] 2. Terminal data collection and transmission

[2277] The terminal collects the user's or robot's identification information and voice data and sends it to the server. The identification information is automatically read using a radio frequency identification tag (RFID) or barcode reader. The identification information and voice data acquired by the terminal are encrypted using a secure protocol (e.g., HTTPS) and sent to the server.

[2278] 3. Server Data Analysis

[2279] The server passes the received voice data to a generative artificial intelligence (AI) system, which converts it into text using speech recognition technology. The converted text data is then analyzed using natural language processing (NLP) technology to assess the nature of the anomaly and its urgency.

[2280] 4. Emotion analysis using an emotion engine

[2281] The server passes the analyzed text data to the emotion engine, which then reflects the emotion analysis in assessing the urgency of the anomaly. The emotion engine recognizes emotions from the voice tone, speech content, speech rate, or the robot's movement patterns, and adjusts the urgency accordingly.

[2282] 5. Prioritization

[2283] The server determines the priority of anomalies based on the analysis results of the generative artificial intelligence and emotion engine, and classifies the priority as "high," "medium," or "low."

[2284] 6. Notification to staff

[2285] The server sends notifications to the maintenance staff's terminals according to the determined priority. High-priority anomalies are notified immediately, while medium- and low-priority anomalies are notified at an appropriate time. The notification includes the nature of the anomaly, priority, and emotional state.

[2286] 7. Staff Response and Reporting

[2287] The user (maintenance staff) receives the notification from the terminal and begins to respond based on priority. When the staff member starts to respond, they input "response start" into the terminal. A timestamp of the response start is sent to the server and the response log is updated. When the response is completed, they input "response completion" into the terminal and a completion timestamp is sent to the server. The server receives the response completion timestamp, updates the response history, and records the abnormality processing status.

[2288] Specific examples

[2289] As example 1, we will introduce a scenario in which an emergency robot abnormality occurs.

[2290] Robot ID: robot_123

[2291] Error message: Critical failure, motor overheating.

[2292] Analyze the details of the error message with generative AI and use an emotion engine to assess urgency and display the emotional state.

[2293] The server analyzes this error message, rates it as high urgency, and sends an immediate notification to maintenance staff.

[2294] The above configuration enables prompt and appropriate response when an abnormality occurs in a robot in a factory, thereby reducing the burden on maintenance staff and improving factory production efficiency.

[2295] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2296] Step 1:

[2297] User abnormality notification begins

[2298] When an abnormality occurs in a robot in a factory, the user can report the abnormality by pressing a dedicated button on the terminal or by voice. The input is a voice message or button operation when an abnormality occurs. The output is a signal notifying the terminal of the abnormality. This causes the system to start receiving abnormality notifications.

[2299] Step 2:

[2300] Collection of identification information and voice data

[2301] The terminal automatically reads the user's or robot's identification information using an RFID or barcode reader, and also collects voice messages from the user as voice data. The input is the RFID tag or barcode and the voice message. The output is the identification information and voice data.

[2302] Step 3:

[2303] Sending data to the server

[2304] The device sends the collected identification information and voice data to the server using a secure protocol (e.g., HTTPS). The input is the identification information and voice data. The output is the data sent to the server.

[2305] Step 4:

[2306] Analysis of audio data

[2307] The server passes the received voice data to a generative AI, which converts the voice data into text using speech recognition technology. The input is the voice data, and the output is the converted text data. A generative AI model (e.g., GPT-4) is used for this process.

[2308] Step 5:

[2309] Text data analysis

[2310] The server analyzes the converted text data using a natural language processing (NLP) tool to evaluate the nature of the anomaly and its urgency. The input is the text data. The output is the analysis results, which include the nature of the anomaly and its urgency. An NLP tool (e.g., SpaCy) is used for this process.

[2311] Step 6:

[2312] Emotion Analysis

[2313] The server passes the analyzed text data to an emotion engine, which recognizes emotions from the voice tone and content of the speech. The input is text data. The output is the emotional state and urgency adjustment information as the result of emotion analysis. An emotion engine (e.g., IBM Watson Tone Analyzer) is used for this process.

[2314] Step 7:

[2315] Determining priorities

[2316] The server determines the priority of anomalies based on the analysis results of the generative artificial intelligence and emotion engine. The inputs are the details of the anomaly, the urgency, and the emotion analysis results. The output is the determined priority. Priorities are classified as "high," "medium," or "low."

[2317] Step 8:

[2318] Sending notifications to staff

[2319] The server sends a notification to the maintenance staff's terminal based on the determined priority. The input is the determined priority, the abnormality details, and the staff's emotional state. The output is a notification to the staff's terminal. The notification includes the abnormality details, priority, and emotional state.

[2320] Step 9:

[2321] Staff begins to respond

[2322] The user (maintenance staff) receives the notification and starts a response based on the priority. When the response starts, a timestamp of the response start is sent from the terminal to the server. The input is the notification and the start time, and the output is a response start log.

[2323] Step 10:

[2324] Response completion report

[2325] When the user (maintenance staff) completes the response, they input the response completion into the terminal and send a completion timestamp to the server. The input is the response completion time, and the output is a response completion log. The server receives the response completion timestamp, updates the response history, and records the abnormality processing status.

[2326] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data...

Claims

1. means for a user to initiate a nurse call at a terminal; means for the terminal to transmit patient identification information and voice data to a server; A means for the server to analyze the voice data using generative artificial intelligence and evaluate the content and urgency of the request; A means for the server to determine priority based on urgency; A means for the server to send notifications to the nurse's terminal according to priority; A means for nurses to report the response status to the server via their terminals; A system including:

2. The system of claim 1 , wherein the generative artificial intelligence converts voice data into text using speech recognition technology and performs natural language processing.

3. 10. The system of claim 1, wherein the patient's identification information is obtained using a radio frequency identification tag or a bar code.

Images