System

Abstract

A system is provided.SOLUTION: The system includes a means for registering patient information in a database, a means for acquiring patient identification information by voice recognition, a means for acquiring corresponding patient information from the database on the basis of the patient identification information, and a means for voice-outputting or display-displaying the acquired patient information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In nursing homes and other care facilities, staff need to have detailed information about each elderly person in order to provide appropriate care. However, it is difficult for new or inexperienced staff to quickly recognize and remember this information, which puts them at risk of providing incorrect care. Furthermore, insufficient information sharing makes it difficult for staff to cooperate, resulting in reduced work efficiency. [Means for solving the problem]

[0005] In order to solve the above-mentioned problems, the present invention provides the following means: A system including a means for registering patient information in a database, a means for acquiring patient identification information through voice recognition, a means for acquiring corresponding patient information from the database based on the patient identification information, and a means for outputting the acquired patient information by voice or displaying it on a screen. The system further includes a means for converting voice-input patient condition information into text and updating the corresponding patient information in the database, and a means for processing the voice recognition results in real time and transmitting the acquired patient information to a wearable device, thereby enabling even new staff to quickly grasp a patient's condition and provide appropriate care.

[0006] "Means for registering patient information in a database" refers to a means that has the function of electronically recording information about a patient, such as name, ID, medical history, recent condition, and precautions, and storing it in a database.

[0007] The "means for obtaining patient identification information by voice recognition" refers to a means having the function of processing voice input and extracting patient identification information (e.g., name, ID) as text data.

[0008] The "means for obtaining relevant patient information from a database based on patient identification information" refers to a means having a function for searching a database using patient identification information and extracting corresponding patient information.

[0009] The "means for outputting acquired patient information as voice or displaying it on a display" refers to a means having a function for outputting acquired patient information as voice or displaying it on a display.

[0010] "Means for converting voice-inputted patient condition information into text and updating the corresponding patient information in the database" refers to means for converting new patient condition information inputted by voice into text data and using that data to update the corresponding patient information in the database.

[0011] "Means for processing voice recognition results in real time and transmitting the acquired patient information to a wearable terminal" refers to means that has the function of instantly processing voice recognition results and transmitting the processed information to a wearable terminal. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0013] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0016] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0017] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0018] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0020] [First embodiment]

[0021] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0022] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0025] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0028] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0032] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0033] The present invention relates to an information sharing system that enables staff in nursing homes and other care facilities to efficiently provide appropriate care to individual elderly people. The following describes in detail how the system of the present invention operates.

[0034] System Overview

[0035] The system of the present invention comprises the following elements:

[0036] 1. Database: Stores information such as patient name, ID, medical history, recent condition, and notes.

[0037] 2. Voice recognition device: Converts patient identification information entered by staff into text data.

[0038] 3. Information acquisition device: Extracts relevant patient information from the database based on the converted text data.

[0039] 4. Audio output and display devices: Provide retrieved patient information to staff.

[0040] 5. Updater: Converts new patient status information entered by voice into text and updates the database.

[0041] Program processing

[0042] Patient information registration

[0043] 1. The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions using a dedicated input form.

[0044] 2. The terminal packages the entered information and sends it to the server.

[0045] 3. The server stores the received information in the database, creating a new record for a new patient and updating the existing record for an existing patient.

[0046] Searching and retrieving patient information

[0047] 1. The user (staff member) speaks the patient's identification information (e.g., ID1234) into the wearable device.

[0048] 2. The device converts the voice input into text data in real time and sends the text data to the server.

[0049] 3. The server searches the database based on the received text data and retrieves the relevant patient information.

[0050] 4. The server sends the acquired patient information to the terminal in voice or text format.

[0051] 5. The terminal outputs the received patient information by voice or displays it on the display.

[0052] Patient Information Updates

[0053] 1. The user (staff member) speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID 1234").

[0054] 2. The device converts the voice into text data and sends it to the server.

[0055] 3. The server updates the relevant patient information in the database based on the text data received.

[0056] Specific examples

[0057] Example 1: Retrieving patient information

[0058] 1. The user (staff member) speaks into the wearable device, "Please tell me the information for ID1234."

[0059] 2. The device converts the voice to text and sends the data "Please tell me the information about ID1234" to the server.

[0060] 3. The server retrieves the patient information corresponding to "ID1234" from the database.

[0061] 4. The server converts the acquired information into audio data and sends it to the device.

[0062] 5. The device outputs a voice message saying, "Yamada Taro has diabetes. His blood sugar levels have been stable recently. He needs to be careful about his diet."

[0063] Example 2: Updating patient information

[0064] 1. The user (staff member) speaks into the wearable device, "Record the recent status of ID1234."

[0065] 2. The device converts the voice to text and sends the data "Record the recent status of ID1234" to the server.

[0066] 3. The device will say, "Status has been checked. Please tell us the latest status."

[0067] 4. The user (staff member) speaks, "My blood sugar level was high yesterday."

[0068] 5. The device converts the voice into text and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[0069] 6. The server updates the relevant patient information in the database.

[0070] As described above, the system of the present invention enables nursing home staff to provide prompt and appropriate care to each elderly person.

[0071] The processing flow will be explained below.

[0072] Patient information database registration

[0073] Step 1:

[0074] The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions into a dedicated form.

[0075] Step 2:

[0076] The terminal converts the input information into a data package and transmits it to the server.

[0077] Step 3:

[0078] The server analyzes the received information and stores it in the database. For new patients, it creates a new record, and for existing patients, it updates the corresponding record.

[0079] Searching and retrieving patient information

[0080] Step 1:

[0081] The user (staff member) speaks the patient's identification information (e.g., ID1234) into the wearable device.

[0082] Step 2:

[0083] The device converts the voice input into text data in real time and sends the text data to the server.

[0084] Step 3:

[0085] The server searches the database based on the received text data and retrieves the relevant patient information.

[0086] Step 4:

[0087] The server converts the acquired patient information into text or voice data and sends it to the terminal.

[0088] Step 5:

[0089] The patient information received by the terminal is output as voice or displayed on the screen.

[0090] Patient Information Updates

[0091] Step 1:

[0092] The user (staff member) speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID 1234").

[0093] Step 2:

[0094] The device converts the voice into text data and sends it to the server.

[0095] Step 3:

[0096] The device will output a voice message saying, "Status has been checked. Please tell us the latest status."

[0097] Step 4:

[0098] The user (staff member) inputs by voice, "My blood sugar level was high yesterday."

[0099] Step 5:

[0100] The device converts the voice into text data, creating text data such as "yesterday's blood sugar level was high."

[0101] Step 6:

[0102] The terminal transmits the text data to the server.

[0103] Step 7:

[0104] The server updates the relevant patient information in the database.

[0105] Example 1

[0106] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0107] Conventional patient information management systems in nursing care facilities require staff to manually input and manage patient information, placing a heavy burden on staff and taking time to search for and update information.In addition, since information cannot be obtained or updated in real time via voice input, efficient information management is not possible in emergencies or when staff are busy.

[0108] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0109] In this invention, the server includes means for registering patient information in a database, means for acquiring patient identification information by voice recognition, means for acquiring corresponding patient information from the database based on the patient identification information, means for converting voice-input patient condition information into text and updating the corresponding patient information in the database, and means for processing the voice recognition results in real time and transmitting the acquired patient information to the wearable device, thereby enabling staff to efficiently manage patient information using voice input and acquire and update information in real time.

[0110] "Means for registering patient information in a database" refers to a system for storing information such as the patient's name, ID, medical history, recent condition, and precautions in a database.

[0111] The "means for acquiring patient identification information by voice recognition" is a mechanism for recognizing patient identification information input by voice, converting it into text data, and acquiring it.

[0112] The "means for obtaining relevant patient information from a database based on patient identification information" refers to a mechanism for searching a database based on identification information obtained by voice recognition and obtaining corresponding patient information.

[0113] "Means for outputting acquired patient information by voice or displaying it on a screen" refers to a system for conveying patient information acquired from a database to staff through voice or a display.

[0114] "Means for converting voice-input patient condition information into text and updating the corresponding patient information in the database" refers to a system in which staff converts new patient condition information entered by voice into text and updates the corresponding information in the database based on that information.

[0115] "Means for processing voice recognition results in real time and transmitting acquired patient information to a wearable device" refers to a mechanism for instantly processing the results obtained by voice recognition and transmitting acquired patient information based on the results to a wearable device.

[0116] The present invention relates to an information sharing system that enables staff in a nursing home to efficiently provide appropriate care to individual patients. The operation of the system of the present invention will be specifically described below.

[0117] System Overview

[0118] The system consists of the following elements:

[0119] 1. Database: Stores information such as patient name, ID, medical history, recent condition, and notes.

[0120] 2. Voice recognition device: Converts patient identification information entered by staff into text data using the Google® Cloud Speech-to-Text API.

[0121] 3. Information acquisition device: Extracts relevant patient information from the database based on the converted text data.

[0122] 4. Audio output and display devices: Provide acquired patient information to staff. Use Web Speech API for audio output and a standard tablet or smartphone for display.

[0123] 5. Updater: Converts new patient status information entered by voice into text and updates the database.

[0124] Program processing

[0125] Patient information registration

[0126] 1. The user (administrator) uses a dedicated input form to enter information such as the patient's name, ID, medical history, recent condition, and precautions.

[0127] 2. The terminal packages the entered information and sends it to the server.

[0128] 3. The server stores the received information in the database, creating a new record for a new patient and updating the existing record for an existing patient.

[0129] Searching and retrieving patient information

[0130] 1. The user (staff member) speaks the patient's identification information into the wearable device.

[0131] 2. The device converts the voice input into text data in real time and sends the text data to the server.

[0132] 3. The server searches the database based on the received text data and retrieves the relevant patient information.

[0133] 4. The server sends the acquired patient information to the terminal in voice or text format.

[0134] 5. The terminal outputs the received patient information by voice or displays it on the display.

[0135] Patient Information Updates

[0136] 1. The user (staff member) speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID 1234").

[0137] 2. The device converts the voice into text data and sends it to the server.

[0138] 3. The server updates the relevant patient information in the database based on the text data received.

[0139] Specific examples

[0140] Example 1: Retrieving patient information

[0141] 1. The user (staff member) speaks into the wearable device, "Please tell me the information for ID1234."

[0142] 2. The device converts the voice to text and sends the data "Please tell me the information about ID1234" to the server.

[0143] 3. The server retrieves the patient information corresponding to "ID1234" from the database.

[0144] 4. The server converts the acquired information into audio data and sends it to the device.

[0145] 5. The device outputs a voice message saying, "Yamada Taro has diabetes. His blood sugar levels have been stable recently. He needs to be careful about his diet."

[0146] Example 2: Updating patient information

[0147] 1. The user (staff member) speaks into the wearable device, "Record the recent status of ID1234."

[0148] 2. The device converts the voice to text and sends the data "Record the recent status of ID1234" to the server.

[0149] 3. The device will say, "Status has been checked. Please tell us the latest status."

[0150] 4. The user (staff member) speaks, "My blood sugar level was high yesterday."

[0151] 5. The device converts the voice into text and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[0152] 6. The server updates the relevant patient information in the database.

[0153] As described above, the system of the present invention enables nursing home staff to obtain and update patient information in real time, enabling them to provide prompt and appropriate care.

[0154] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0155] Patient information registration

[0156] Step 1:

[0157] The user enters patient information using an input form. The information to be entered includes the patient's name, ID, medical history, recent condition, and precautions. Specifically, the user enters each item into the form on the web browser and clicks the submit button. The input data at this point is the string data for each item entered by the user.

[0158] Step 2:

[0159] The device packages the input information and sends it to the server. Specifically, JavaScript (registered trademark) running on the device converts the input data into JSON format and sends it to the server via an HTTP POST request. The input is patient information in JSON format, and the output is an HTTP request.

[0160] Step 3:

[0161] The server stores the received information in the database. During this process, the server parses the HTTP POST request and extracts the JSON data. For new patients, it creates a new record using the SQL INSERT statement, and for existing patients, it updates the record using the UPDATE statement. The input is JSON-formatted data, and the output is the patient information stored in the database.

[0162] Searching and retrieving patient information

[0163] Step 1:

[0164] The user speaks the patient's identification information into the wearable device. Specifically, the user presses the voice input button and says, "Tell me the information about ID1234." The input data at this point is voice data.

[0165] Step 2:

[0166] The device converts voice input into text data in real time and sends that text data to the server. Specifically, a speech recognition engine (e.g., Google Cloud Speech-to-Text API) analyzes the voice data and generates text data such as "Please tell me the information about ID1234." This text data is then sent to the server as an HTTP request. The input is voice data, and the output is text data and an HTTP request.

[0167] Step 3:

[0168] The server searches the database based on the received text data and retrieves the corresponding patient information. Specifically, the server analyzes the text data and retrieves the patient information from the database using an SQL SELECT statement. The input is text data, and the output is the corresponding patient information.

[0169] Step 4:

[0170] The server acquires patient information and sends it to the terminal in voice or text format. The server converts the acquired information into JSON format and sends it to the terminal as an HTTP response. The input is patient information and the output is an HTTP response.

[0171] Step 5:

[0172] The terminal outputs the received patient information by voice or displays it on the display. Specifically, it uses a voice output library (e.g., SpeechSynthesis of the Web Speech API) to convey the information to the user by voice, or displays it as text on the display. The input is patient information in JSON format, and the output is voice data or display.

[0173] Patient Information Updates

[0174] Step 1:

[0175] The user speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID1234"). The input data is voice data.

[0176] Step 2:

[0177] The device converts the voice into text data and sends it to the server. Specifically, the voice recognition engine analyzes the voice data, generates text data such as "Record the recent status of ID1234," and sends it to the server as an HTTP request. The input is voice data, and the output is text data and an HTTP request.

[0178] Step 3:

[0179] The device will output a voice message saying, "The status has been checked. Please tell us the latest status." Specifically, a voice output library is used to play a voice message prompting the user for confirmation. The input is the text data mentioned earlier, and the output is a voice confirmation.

[0180] Step 4:

[0181] The user inputs by voice, "My blood sugar level was high yesterday." The input data is voice data.

[0182] Step 5:

[0183] The device again converts the voice into text data and sends it to the server. Specifically, the voice recognition engine generates text data such as "yesterday's blood sugar level was high" and sends it to the server as an HTTP request. The input is voice data, and the output is text data and an HTTP request.

[0184] Step 6:

[0185] The server updates the corresponding patient information in the database. Specifically, it uses the SQL UPDATE statement to update the corresponding database record based on the text data received by the server. The input is the text data, and the output is the updated database record.

[0186] (Application example 1)

[0187] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0188] In traditional nursing homes and factories, staff were required to efficiently obtain information about patients and equipment and respond appropriately. However, registering and updating information was time-consuming and there was a high possibility of input errors. In addition, it was difficult to obtain the necessary information in real time, which sometimes resulted in delayed or incorrect responses. This led to issues such as reduced operational efficiency at nursing homes and factories and inadequate management of patients and equipment.

[0189] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0190] In this invention, the server includes a means for registering patient information in a database, a means for acquiring patient identification information through voice recognition, a means for registering the operating status of multiple devices in the database, and a means for acquiring device identification information through voice recognition. This allows staff to quickly acquire patient and device information and respond appropriately. Furthermore, by enabling the registration and updating of information through voice recognition, it is possible to prevent input errors and improve operational efficiency.

[0191] "Patient information" refers to information such as the patient's name, ID, medical history, recent condition, and precautions.

[0192] A "database" is a collection of information designed to efficiently store specific information and to make it easy to search, update, and delete.

[0193] "Speech recognition" is a technology that converts speech into text data, analyzes speech input, and outputs a corresponding string of characters.

[0194] "Identification information" refers to information that uniquely identifies a specific individual or device, and includes an ID, name, etc.

[0195] "Audio output" is a technology that outputs audio data in a format that humans can hear through devices such as speakers.

[0196] "Display" is a technology for displaying information such as text and images on the screen of a display device.

[0197] "Device information" refers to information including the operating status of machines and robots, error information, maintenance information, and the like.

[0198] A "wearable device" is a computing device that can be worn by a user, including smart glasses and head-mounted displays.

[0199] The present invention relates to an information sharing system that enables staff in nursing homes and other care facilities, as well as in factories, to efficiently provide appropriate care and maintenance for individual patients and devices. The operation of the system of the present invention will be specifically described below.

[0200] System Overview

[0201] The system of the present invention comprises the following elements:

[0202] 1. Database

[0203] Stores patient and device information. Patient information includes name, ID, medical history, recent condition, and precautions, while device information includes operating status, error information, and maintenance information.

[0204] 2. Voice Recognition Device

[0205] The system converts patient and device identification information entered by staff via voice into text data using Python and SpeechRecognition (Google Speech-to-Text API).

[0206] 3. Information acquisition device

[0207] Based on the converted text data, the relevant patient and device information is extracted from the database using SQLite3.

[0208] 4. Audio output devices and display devices

[0209] The acquired patient and device information is provided to staff using speakers for audio output and tablets or smart glasses for display.

[0210] 5. Update device

[0211] Converts new spoken patient and device status information into text and updates the database. Uses Python, SpeechRecognition, and SQLite3.

[0212] Program processing

[0213] Patient information registration

[0214] The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions using a dedicated input form, and the terminal packages the entered information and sends it to the server. The server stores the received information in a database. At this time, a new record is created for new patients, and the corresponding record is updated for existing patients.

[0215] Registering device information

[0216] The user (administrator) enters information such as the device ID, operating status, error information, and maintenance information using a dedicated input form, and the terminal packages the entered information and sends it to the server. The server stores the received information in a database. At this time, a new record is created for new devices, and the corresponding record is updated for existing devices.

[0217] Search and retrieve patient and device information

[0218] The user (staff member) speaks into the wearable terminal to input the patient or device identification information, which the terminal converts into text data and sends to the server. The server searches the database based on the received text data and obtains the relevant patient and device information. The server then sends the obtained information to the terminal in voice or text format, and the terminal outputs the received information as voice or displays it on a display.

[0219] Specific examples

[0220] Example 1: Retrieving patient information

[0221] The user (staff member) speaks into the wearable device, saying, "Tell me the information about patient ID 1234." The device converts the speech into text and sends the data "Tell me the information about patient ID 1234" to the server. The server retrieves the patient information corresponding to "patient ID 1234" from the database. The server converts the retrieved information into voice data and sends it to the device. The device outputs a voice message saying, "Patient Name has diabetes. His blood sugar level has been stable recently. He needs to be careful about his diet."

[0222] Example 2: Updating Device Information

[0223] The user (staff member) speaks to the wearable device, saying, "The status of device ID 002 will be updated. The new status is an error, and the details are motor failure." The device converts the speech into text and sends the data "The status of device ID 002 will be updated. The new status is an error, and the details are motor failure" to the server. The server updates the relevant device information in the database, and the device outputs a voice message saying, "The status has been updated."

[0224] Prompt Sentence Examples

[0225] 1. "Please check the status of the following machine: Machine ID: 12345"

[0226] 2. "Please update the status of machine ID: 12345. New status: Under maintenance, Details: Part replaced"

[0227] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0228] Step 1:

[0229] The user inputs the patient or device identification information (e.g., patient ID or device ID) by voice. The input voice data is picked up by the microphone. The picked up voice data is sent by the terminal to the voice recognition engine (SpeechRecognition). The voice recognition engine converts the voice data into text data and outputs the text data to the terminal.

[0230] Input: Voice data (patient or device identification information)

[0231] Output: Text data (converted identification information)

[0232] Step 2:

[0233] The device receives the text data, packages it, and sends it to the server. The device converts the text data to JSON format and sends it as an HTTP request to the server. The server receives the HTTP request and prepares a database query.

[0234] Input: Text data of identification information

[0235] Output: HTTP request sent to the server

[0236] Step 3:

[0237] The server searches the database (SQLite3) based on the received text data and retrieves the corresponding patient or device information. The server executes an SQL query and retrieves the corresponding records. The retrieved information is converted to JSON format on the server and sent to the device as an HTTP response.

[0238] Input: The identification information sent to the server

[0239] Output: JSON data of the relevant patient or device information

[0240] Step 4:

[0241] The device analyzes the JSON data received from the server and provides information to the user through a voice output device or display device. The device parses the JSON data and converts it into a format for a speech synthesis engine or display. The speech synthesis engine converts the text data into audio data and plays it over the speaker. The display device displays the formatted text.

[0242] Input: JSON data from the server

[0243] Output: Audio output or display

[0244] Step 5:

[0245] When the user speaks a new patient or device status, the device again converts the voice to text and sends it to the server. The device then uses a speech recognition engine to convert the voice data to text, generates JSON data containing the updated information, and sends it to the server.

[0246] Input: Audio data (new state information)

[0247] Output: HTTP request sent to the server

[0248] Step 6:

[0249] The server updates the corresponding patient or device record in the database based on the received text data. The server executes the SQL UPDATE statement to update the corresponding record in the database with the new state. After the update is complete, the server returns a notification of update completion to the terminal.

[0250] Input: Text data of the update contents

[0251] Output: Notification of update completion

[0252] Step 7:

[0253] The device receives the update completion notification from the server and notifies the user by voice or display that the update is complete. A voice synthesis engine is used to play a message indicating that the update is complete, or a message is displayed on the screen.

[0254] Input: Notification of update completion from the server

[0255] Output: Update completion notification via audio output or display

[0256] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0257] This invention relates to an information sharing system that enables staff in nursing homes and other care facilities to efficiently provide appropriate care to each elderly person. By combining this system with an emotion engine that recognizes the user's emotions, it becomes possible to provide even more accurate care.

[0258] System Overview

[0259] The system of the present invention comprises the following elements:

[0260] 1. Database: Stores information such as patient name, ID, medical history, recent condition, and notes.

[0261] 2. Voice recognition device: Converts patient identification information entered by staff into text data.

[0262] 3. Information acquisition device: Extracts relevant patient information from the database based on the converted text data.

[0263] 4. Audio output and display devices: Provide retrieved patient information to staff.

[0264] 5. Updater: Converts new patient status information entered by voice into text and updates the database.

[0265] 6. Emotion Engine: Recognizes user emotions in real time and adjusts the information provided as needed.

[0266] Program processing

[0267] Patient information registration

[0268] 1. The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions into a dedicated form.

[0269] 2. The terminal packages the input information into a data package and sends it to the server.

[0270] 3. The server analyzes the received information and stores it in the database, creating a new record for a new patient and updating the existing record for an existing patient.

[0271] Searching and retrieving patient information

[0272] 1. The user (staff member) speaks the patient's identification information (e.g., ID1234) into the wearable device.

[0273] 2. The device converts the voice input into text data in real time and sends the text data to the server.

[0274] 3. The server searches the database based on the received text data and retrieves the relevant patient information.

[0275] 4. The server converts the acquired patient information into text or voice data and sends it to the terminal.

[0276] 5. The terminal outputs the received patient information by voice or displays it on the display.

[0277] Patient Information Updates

[0278] 1. The user (staff member) speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID 1234").

[0279] 2. The device converts the voice into text data and sends it to the server.

[0280] 3. The device will say, "Status has been checked. Please tell us the latest status."

[0281] 4. The user (staff member) speaks, "My blood sugar level was high yesterday."

[0282] 5. The device converts the voice into text data and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[0283] 6. The server updates the relevant patient information in the database.

[0284] Recognizing and modulating emotional information

[0285] 1. The device captures the user's (staff member's) voice and facial expressions in real time and sends them to the emotion engine.

[0286] 2. The emotion engine recognizes the user's (staff's) emotions from the received data and captures them as text or metadata.

[0287] 3. The emotion engine adjusts the content and expression of the patient information sent from the server based on the emotion.

[0288] 4. The server sends the adjusted patient information to the terminal and provides it to the staff in an appropriate format.

[0289] Specific examples

[0290] Example 1: Retrieving patient information

[0291] 1. The user (staff member) speaks into the wearable device, "Please tell me the information for ID1234."

[0292] 2. The device converts the voice to text and sends the data "Please tell me the information about ID1234" to the server.

[0293] 3. The server retrieves the patient information corresponding to "ID1234" from the database.

[0294] 4. The emotion engine analyzes the user's (staff's) current emotions and adjusts the way information is presented as needed (e.g., if the user is feeling stressed, it summarizes the output and presents it accordingly).

[0295] 5. The server converts the adjusted information into audio data and sends it to the device.

[0296] 6. The device outputs a voice message saying, "Yamada Taro has diabetes. His blood sugar levels have been stable recently. He needs to be careful about his diet."

[0297] Example 2: Updating patient information

[0298] 1. The user (staff member) speaks into the wearable device, "Record the recent status of ID1234."

[0299] 2. The device converts the voice to text and sends the data "Record the recent status of ID1234" to the server.

[0300] 3. The device will say, "Status has been checked. Please tell us the latest status."

[0301] 4. The user (staff member) speaks, "My blood sugar level was high yesterday."

[0302] 5. The device converts the voice into text and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[0303] 6. The sentiment engine analyzes the user's emotions and asks additional questions to ensure the input is accurate (e.g., "Was it really expensive?").

[0304] 7. The server updates the patient information in the database.

[0305] As described above, the system of the present invention enables nursing home staff to provide prompt and appropriate care to each elderly person, and by incorporating the user's emotions, the accuracy of information and the quality of care can be improved.

[0306] The processing flow will be explained below.

[0307] Patient information database registration

[0308] Step 1:

[0309] The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions into a dedicated form.

[0310] Step 2:

[0311] The terminal converts the input information into a data package and transmits it to the server.

[0312] Step 3:

[0313] The server analyzes the received information and stores it in the database. For new patients, it creates a new record, and for existing patients, it updates the corresponding record.

[0314] Searching and retrieving patient information

[0315] Step 1:

[0316] The user (staff member) speaks the patient's identification information (e.g., ID1234) into the wearable device.

[0317] Step 2:

[0318] The device converts the voice input into text data in real time and sends the text data to the server.

[0319] Step 3:

[0320] The server searches the database based on the received text data and retrieves the relevant patient information.

[0321] Step 4:

[0322] The device captures the user's (staff member's) voice and facial expressions and sends them to the emotion engine.

[0323] Step 5:

[0324] The emotion engine recognizes the user's (staff's) emotions from the received data and generates emotion information.

[0325] Step 6:

[0326] The server adjusts the acquired patient information based on the emotion information and summarizes the information as necessary.

[0327] Step 7:

[0328] The server converts the adjusted patient information into voice or text data and transmits it to the terminal.

[0329] Step 8:

[0330] The patient information received by the terminal is output as voice or displayed on the screen.

[0331] Patient Information Updates

[0332] Step 1:

[0333] The user (staff member) speaks into the wearable device, "Record the recent status of ID1234."

[0334] Step 2:

[0335] The device converts the voice into text data and sends it to the server.

[0336] Step 3:

[0337] The device will output a voice message saying, "Status has been checked. Please tell us the latest status."

[0338] Step 4:

[0339] The user (staff member) inputs by voice, "My blood sugar level was high yesterday."

[0340] Step 5:

[0341] The device converts the voice into text data, generating text data such as "yesterday's blood sugar level was high."

[0342] Step 6:

[0343] The terminal transmits the text data to the server.

[0344] Step 7:

[0345] The server analyzes the text data and updates the relevant patient information in the database.

[0346] Step 8:

[0347] The device captures the user's (staff member's) voice and facial expressions and sends them to the emotion engine.

[0348] Step 9:

[0349] The emotion engine recognizes the user's (staff's) emotions from the received data and adds questions to confirm the accuracy of the input.

[0350] Step 10:

[0351] The terminal transmits the re-entered information to the server.

[0352] Step 11:

[0353] The server will then finalize the patient information in the database and update it if necessary.

[0354] Example 2

[0355] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0356] Conventional information sharing systems in nursing homes mainly involve managing patient information manually, which often lacks efficiency and accuracy. Furthermore, information is provided without taking into account the emotions and stress levels of users (staff), making it impossible to improve the quality of care. It is necessary to provide a system that solves these problems and provides prompt and appropriate care to each elderly person.

[0357] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0358] In this invention, the server includes means for registering patient information in a database, means for acquiring patient identification information by voice recognition, means for acquiring corresponding patient information from the database based on the patient identification information, means for outputting the acquired patient information by voice or displaying it on a screen, means for adjusting the content and presentation method of information using an emotion engine that recognizes user emotions, means for converting voice-input patient condition information into text and updating the corresponding patient information in the database, means for processing the voice recognition results in real time and transmitting the acquired patient information to a wearable device, means for capturing the user's voice and facial expressions and transmitting them to the emotion engine, and means for adjusting the voice or text output based on the user's emotions. This allows information to be registered, acquired, and updated in the database accurately and efficiently, and by taking the user's emotions into consideration, it is possible to improve the quality of care.

[0359] The "means for registering patient information in a database" is a component that performs processing to store information such as the patient's name, ID, medical history, recent condition, and points to note in a database system.

[0360] The "means for acquiring patient identification information by voice recognition" is a mechanism for acquiring patient identification information input by voice as text data using voice recognition technology.

[0361] The "means for acquiring corresponding patient information from a database based on patient identification information" is a component that performs processing to search for and extract corresponding patient information from within a database using the acquired identification information.

[0362] "Means for outputting acquired patient information by voice or displaying it on a screen" refers to a mechanism for outputting or displaying information by voice or on a screen in order to convey searched or acquired patient information to the user.

[0363] The "emotion engine that recognizes user emotions" is a technology and software that analyzes and recognizes emotions from the user's voice and facial expressions in real time.

[0364] The "means for converting voice-inputted patient condition information into text and updating the corresponding patient information in the database" is a component that converts voice-inputted information into text and performs the process of updating the patient information stored in the database to the latest state.

[0365] The "means for processing voice recognition results in real time and transmitting the acquired patient information to a wearable terminal" is a mechanism for processing text data obtained by voice recognition in real time and transmitting the corresponding patient information to a wearable terminal.

[0366] The "means for capturing the user's voice and facial expression and transmitting it to the emotion engine" is a system for capturing the user's voice and facial expression and transmitting that data to the emotion engine.

[0367] "Means for adjusting voice or text output based on user emotions" refers to technology for adjusting the content and expression of the voice or text to be output in accordance with the user's emotions recognized by the emotion engine.

[0368] This invention relates to an information sharing system that enables staff in nursing homes and other care facilities to efficiently provide appropriate care to each elderly person. This system registers patient information in a database, acquires patient identification information using voice recognition technology, retrieves relevant patient information from the database based on that information, and outputs the acquired patient information as voice output or on a display. Furthermore, it differs from conventional systems in that it can recognize the user's emotions and adjust the content and presentation method of information based on those emotions.

[0369] The main components of the system are as follows:

[0370] Hardware and Software Components

[0371] 1. Database:

[0372] It stores information such as the patient's name, ID, medical history, recent condition, and precautions. As a specific example, a relational database such as MySQL (registered trademark) is used.

[0373] 2. Voice recognition device:

[0374] The patient's identification information is converted into text data by staff using the Google Cloud Speech-to-Text API.

[0375] 3. Information acquisition device:

[0376] Based on the converted text data, the relevant patient information is extracted from the database using a script (e.g., Python) that runs on the server.

[0377] 4. Audio output and display devices:

[0378] Use voice output devices and display terminals to provide retrieved patient information to staff. Examples include smartwatches and tablets.

[0379] 5. Update device:

[0380] New patient status information is spoken into the database and converted to text, using the Google Cloud Speech-to-Text API as well as a speech recognizer.

[0381] 6. Emotion Engine:

[0382] It recognizes the user's emotions in real time and adjusts the information provided as needed. As a concrete example, it uses the Microsoft® Azure® Emotion API.

[0383] Examples of concrete examples and prompts

[0384] 1. Obtaining patient information

[0385] The user (staff member) speaks into the wearable device, "Please tell me the information for ID1234."

[0386] The device converts the voice into text and sends the data "Please tell me the information about ID1234" to the server.

[0387] The server retrieves the patient information corresponding to "ID1234" from the database.

[0388] The emotion engine analyzes the user's (staff's) current emotions and adjusts the way information is presented as needed (e.g., if the user is feeling stressed, it summarizes the output and presents it accordingly).

[0389] The server converts the adjusted information into audio data and transmits it to the terminal.

[0390] The device outputs a voice message saying, "Yamada Taro has diabetes. His blood sugar levels have been stable recently. He needs to be careful about his diet."

[0391] 2. Updating patient information

[0392] The user (staff member) speaks into the wearable device, "Record the recent status of ID1234."

[0393] The device converts the voice into text and sends the data "Record the recent status of ID1234" to the server.

[0394] The device will output a voice message saying, "Status has been checked. Please tell us the latest status."

[0395] The user (staff member) inputs by voice, "My blood sugar level was high yesterday."

[0396] The device converts the voice into text and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[0397] The server updates the relevant patient information in the database.

[0398] By using these specific procedures and techniques, the present invention is designed to enable nursing home staff to provide more efficient and accurate care. As described above, the present invention enables the registration, acquisition, and updating of patient information quickly and accurately, and further improves the quality of care by taking into account the user's emotions.

[0399] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0400] Patient information registration

[0401] Step 1:

[0402] The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions into a dedicated form on the browser.

[0403] Specifically, the administrator enters information such as "Yamada Taro," "ID 1234," "has had diabetes for 10 years," "his blood sugar levels are stable these days," and "pay attention to dietary management."

[0404] Step 2:

[0405] The terminal converts the input information into a JSON format data package and sends it to the server.

[0406] Specifically, the device processes the input information into data such as "{"name":"Yamada Taro","ID":"ID1234","Medical history":"Diabetes for 10 years","Recent condition":"Stable blood sugar","Points to note":"Dietary management"}" and sends it to the server.

[0407] Step 3:

[0408] The server analyzes the received JSON data and stores it in the database.

[0409] Specifically, the server inserts a new record into the "patients" table or "updates" an existing record. Based on the JSON data as input, it generates a status indicating the data has been saved to the database as output.

[0410] Searching and retrieving patient information

[0411] Step 1:

[0412] The user (staff member) speaks into the wearable device to input the patient's identification information (e.g., "Please tell me the information for ID1234").

[0413] Specifically, the staff member speaks into the smartwatch, saying, "Tell me the information for ID1234."

[0414] Step 2:

[0415] The device converts the voice input into text data in real time and sends the text data to the server.

[0416] Specifically, the device converts the voice into text data such as "Tell me the information about ID1234" and sends it to the server.

[0417] Step 3:

[0418] The server queries the database based on the received text data and retrieves the corresponding patient information.

[0419] Specifically, the server executes the query "SELECT FROM patients WHERE ID='ID1234'." Based on the text data as input, corresponding patient information is obtained as output.

[0420] Step 4:

[0421] The server processes the acquired patient information into JSON format data and sends it to the terminal.

[0422] Specifically, the server generates the data "{"name":"Yamada Taro","ID":"ID1234","Medical history":"Diabetes for 10 years","Recent condition":"Stable blood sugar","Cautions":"Dietary management"}" and sends it to the terminal.

[0423] Step 5:

[0424] The patient information received by the terminal is output as voice or displayed on the screen.

[0425] Specifically, the device will output a message in voice or display on the screen saying, "Yamada Taro has diabetes. His blood sugar levels have been stable recently. He needs to be careful about his diet."

[0426] Patient Information Updates

[0427] Step 1:

[0428] The user (staff member) speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID 1234").

[0429] As for specific actions, the staff member said, "We will record the recent status of ID1234."

[0430] Step 2:

[0431] The device converts the voice into text data and sends it to the server.

[0432] Specifically, the terminal converts this into text data that reads "Record the recent status of ID1234" and sends it to the server.

[0433] Step 3:

[0434] The device will output a voice message saying, "Status has been checked. Please tell us the latest status."

[0435] Specifically, the device will announce in a voice message, "Status has been checked. Please tell us the latest status."

[0436] Step 4:

[0437] The user (staff member) inputs by voice, "My blood sugar level was high yesterday."

[0438] Specifically, the staff member says, "yesterday's blood sugar level was high."

[0439] Step 5:

[0440] The device converts the voice into text data and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[0441] Specifically, the device converts the text "yesterday's blood sugar level was high" and sends it to the server.

[0442] Step 6:

[0443] The server updates the relevant patient information in the database.

[0444] Specifically, the server executes the query "UPDATE patients SET recent status = 'high blood glucose level' WHERE ID = 'ID1234'". Based on the text data as input, the database is updated as output.

[0445] Recognizing and modulating emotional information

[0446] Step 1:

[0447] The device captures the user's (staff member's) voice and facial expressions in real time and sends them to the emotion engine.

[0448] Specifically, the device uses its built-in camera and microphone to capture the facial expressions and voice tone of the staff member.

[0449] Step 2:

[0450] The emotion engine recognizes the user's (staff's) emotions from the received data and captures them as text or metadata.

[0451] Specifically, the emotion engine generates emotion data such as "feeling stressed."

[0452] Step 3:

[0453] The emotion engine adjusts the content and expression of the patient information sent from the server based on the emotion.

[0454] Specifically, the emotion engine instructs the server to "summarize and present information to reduce stress."

[0455] Step 4:

[0456] The server sends the adjusted patient information to the terminal and provides it to the staff in an appropriate format.

[0457] Specifically, the server sends adjusted information to the terminal, such as "Yamada Taro has diabetes. He needs to be careful."

[0458] The above are the specific processing steps of the system, and the detailed operations and data processing / calculation contents at each step.

[0459] (Application example 2)

[0460] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0461] Nursing care facilities require an information sharing system that enables staff to provide prompt and appropriate care to patients. However, with existing systems, staff fatigue and stress can affect the quality of care, and the method of presenting information is not sufficiently optimized. Furthermore, there is a high risk of input errors when updating information via voice input, making information reliability an issue. This hinders the consistency and efficiency of the quality of care.

[0462] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0463] In this invention, the server includes means for registering patient information in a database, means for acquiring patient identification information by voice recognition, means for acquiring corresponding patient information from the database based on the patient identification information, means for recognizing staff emotions using an emotion engine, means for adjusting the method of providing patient information based on the recognized emotion, means for converting voice-input patient condition information into text and updating the corresponding patient information in the database, and means for processing the voice recognition results in real time and transmitting the acquired patient information to the wearable device. This makes it possible to optimize the method of providing information taking into account the emotions of staff and efficiently provide appropriate care to patients.

[0464] "Patient information" refers to data about individual patients, such as their name, ID, medical history, recent condition, and precautions.

[0465] A "database" is a collection of information that stores patient information in an organized manner and that can be quickly retrieved and updated as needed.

[0466] "Speech recognition" is a technology that converts speech into text data in real time.

[0467] "Patient identification information" refers to information for individually identifying a patient, and includes ID, name, etc.

[0468] "Audio output" is a means for providing acquired information to the user in audio form.

[0469] "Display" refers to a means for visually presenting acquired information to the user.

[0470] An "emotion engine" is a technology that recognizes a user's emotions by analyzing data such as voice and facial expressions.

[0471] A "wearable terminal" is an electronic device that can be worn by a user and is capable of receiving and outputting information.

[0472] "Real time" means that processing occurs the instant an event occurs.

[0473] MODE FOR CARRYING OUT THE INVENTION

[0474] System Configuration

[0475] The present invention is an information sharing system that comprises the following elements:

[0476] 1. Database:

[0477] Stores information such as patient name, ID, medical history, recent status, and precautions.

[0478] 2. Voice recognition device:

[0479] This device converts patient identification information entered by staff into text data. Specifically, smart glasses or wearable devices capture voice and convert it into text in real time.

[0480] 3. Information acquisition device:

[0481] Extract relevant patient information from the database. Search the database based on the text data converted by voice recognition to obtain the necessary patient information.

[0482] 4. Audio output and display devices:

[0483] This device outputs acquired patient information as voice or displays it on a display. It converts text data of patient information into voice data and provides it to the user via a wearable device.

[0484] 5. Update device:

[0485] This system converts new patient status information entered by voice into text and updates the database. Specifically, it has the function of sending the information entered by staff via voice to the server as text data and updating the corresponding record in the database.

[0486] 6. Emotion Engine:

[0487] This engine captures the user's emotions in real time from their voice and facial expressions, and adjusts the way information is presented as needed. This allows it to maintain the quality of care by adjusting the way information is presented if staff are under high stress or fatigue.

[0488] Example of a system

[0489] Example 1: Retrieving patient information

[0490] 1. The user (staff member) speaks into the smart glasses, "Please tell me the patient's ID number."

[0491] 2. The device converts the speech into text and sends it to the server.

[0492] 3. The server retrieves the relevant patient information from the database and analyzes the staff's emotions through the emotion engine.

[0493] 4. The emotion engine assesses whether stress is present and summarizes the information provided if necessary.

[0494] 5. The device outputs the information it has acquired in voice (e.g., "You are a diabetic patient. Your condition has recently stabilized.").

[0495] Example 2: Updating patient information

[0496] 1. The user (staff member) speaks into the smart glasses, "I would like to record the patient's condition."

[0497] 2. The device converts the speech into text and sends it to the server.

[0498] 3. The server retrieves the patient's information from the database and outputs a confirmation message.

[0499] 4. The user (staff member) then voice-inputs, "My blood pressure was high."

[0500] 5. The emotion engine analyzes the user's emotions and confirms the input.

[0501] 6. Update the relevant patient information in the database and notify the patient by voice message that "Information has been updated."

[0502] Hardware and software used

[0503] Hardware:

[0504] Smart glasses: capable of voice input and output

[0505] Microphone: Data acquisition for voice recognition

[0506] Camera: Data acquisition for emotion recognition

[0507] software:

[0508] Python: used as a programming language

[0509] SQLite3: Database Management

[0510] speech_recognition: Speech recognition engine

[0511] pyttsx3: Audio output engine

[0512] emotion_recognition (tentative): Emotion recognition engine

[0513] Prompt Sentence Examples

[0514] Emotion Engine Input:

[0515] python

[0516] emotion_engine.recognize("audio_data_file_path");

[0517] emotion_engine.recognize("video_data_file_path");

[0518] Speech recognition engine prompts:

[0519] python

[0520] recognizer.recognize_google(audio, language='ja-JP');

[0521] Using these technologies, the system provides powerful support for staff to provide patient care quickly and efficiently.

[0522] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0523] Program processing steps

[0524] Example 1: Retrieving patient information

[0525] Step 1:

[0526] The user (staff member) speaks into the smart glasses and says, "Please tell me the patient's ID number."

[0527] Input: Audio data

[0528] How it works: The microphone in the smart glasses captures audio.

[0529] Output: Captured audio data

[0530] Step 2:

[0531] The device calls the speech recognition engine (speech_recognition) to convert the voice data into text data.

[0532] Input: Audio data

[0533] How it works: A speech recognition engine converts voice data into text.

[0534] Output: Text data (e.g., "What is the patient's ID number?")

[0535] Step 3:

[0536] The terminal transmits the text data to the server, and the server retrieves the corresponding patient information from the database.

[0537] Input: Text data

[0538] Operation: Based on the text data, a database is queried to retrieve patient information.

[0539] Output: Patient information data

[0540] Step 4:

[0541] The server uses an emotion engine to recognize the emotions of staff and adjust the way information is presented.

[0542] Input: Patient information data, voice and facial expression data

[0543] How it works: The emotion engine analyzes voice and facial expression data to recognize the user's emotions, adjusting the patient summary and detailed information as needed.

[0544] Output: Reconciled patient information data

[0545] Step 5:

[0546] The terminal calls the voice output engine (pyttsx3) to output the adjusted patient information as voice.

[0547] Input: Adjusted patient information data

[0548] How it works: The voice output engine converts text data into voice data and outputs it from the speaker.

[0549] Output: Speech data (e.g., "I am a diabetic patient. My condition has stabilized recently.")

[0550] Example 2: Updating patient information

[0551] Step 1:

[0552] The user (staff member) speaks into the smart glasses and says, "I would like to record the patient's condition."

[0553] Input: Audio data

[0554] How it works: The microphone in the smart glasses captures audio.

[0555] Output: Captured audio data

[0556] Step 2:

[0557] The device calls the speech recognition engine (speech_recognition) to convert the voice data into text data.

[0558] Input: Audio data

[0559] How it works: A speech recognition engine converts voice data into text.

[0560] Output: Text data (e.g., "I want to record the patient's condition")

[0561] Step 3:

[0562] The terminal sends the text data to the server, which then retrieves the patient's information from the database and outputs a confirmation message.

[0563] Input: Text data

[0564] Behavior: Query database and generate confirmation message based on text data

[0565] Output: Text data of confirmation message

[0566] Step 4:

[0567] The device calls the voice output engine (pyttsx3) to output the confirmation message as voice.

[0568] Input: Text data of confirmation message

[0569] How it works: The voice output engine converts text data into voice data and outputs it from the speaker.

[0570] Output: Audio data (e.g., "We would like to record the patient's status. Do you wish to continue?")

[0571] Step 5:

[0572] The user (staff member) then inputs "My blood pressure was high" by voice.

[0573] Input: Audio data

[0574] How it works: The microphone in the smart glasses captures audio.

[0575] Output: Captured audio data

[0576] Step 6:

[0577] The terminal converts the voice data into text data and sends it to the server.

[0578] Input: Audio data

[0579] How it works: A speech recognition engine converts voice data into text.

[0580] Output: Text data (e.g., "My blood pressure was high")

[0581] Step 7:

[0582] The server uses an emotion engine to analyze the user's emotions and confirm the input content.

[0583] Input: Text data, voice and facial expression data

[0584] How it works: The emotion engine analyzes voice and facial expression data to recognize the user's emotions and generates a confirmation message if necessary.

[0585] Output: Text data of confirmation message (e.g., "Was your blood pressure really high?")

[0586] Step 8:

[0587] The relevant patient information in the database is updated and a voice message is generated saying, "The information has been updated."

[0588] Input: Verified text data

[0589] Behavior: Updates the corresponding record in the database and generates a confirmation message

[0590] Output: Text data of confirmation message

[0591] Step 9:

[0592] The device calls the voice output engine (pyttsx3) to output the confirmation message as voice.

[0593] Input: Text data of confirmation message

[0594] How it works: The voice output engine converts text data into voice data and outputs it from the speaker.

[0595] Output: Audio data (e.g. "Information has been updated")

[0596] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0597] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0598] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0599] [Second embodiment]

[0600] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0601] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0602] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0603] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0604] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0605] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0606] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0607] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0608] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0609] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0610] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0611] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0612] The present invention relates to an information sharing system that enables staff in nursing homes and other care facilities to efficiently provide appropriate care to individual elderly people. The following describes in detail how the system of the present invention operates.

[0613] System Overview

[0614] The system of the present invention comprises the following elements:

[0615] 1. Database: Stores information such as patient name, ID, medical history, recent condition, and notes.

[0616] 2. Voice recognition device: Converts patient identification information entered by staff into text data.

[0617] 3. Information acquisition device: Extracts relevant patient information from the database based on the converted text data.

[0618] 4. Audio output and display devices: Provide retrieved patient information to staff.

[0619] 5. Updater: Converts new patient status information entered by voice into text and updates the database.

[0620] Program processing

[0621] Patient information registration

[0622] 1. The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions using a dedicated input form.

[0623] 2. The terminal packages the entered information and sends it to the server.

[0624] 3. The server stores the received information in the database, creating a new record for a new patient and updating the existing record for an existing patient.

[0625] Searching and retrieving patient information

[0626] 1. The user (staff member) speaks the patient's identification information (e.g., ID1234) into the wearable device.

[0627] 2. The device converts the voice input into text data in real time and sends the text data to the server.

[0628] 3. The server searches the database based on the received text data and retrieves the relevant patient information.

[0629] 4. The server sends the acquired patient information to the terminal in voice or text format.

[0630] 5. The terminal outputs the received patient information by voice or displays it on the display.

[0631] Patient Information Updates

[0632] 1. The user (staff member) speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID 1234").

[0633] 2. The device converts the voice into text data and sends it to the server.

[0634] 3. The server updates the relevant patient information in the database based on the text data received.

[0635] Specific examples

[0636] Example 1: Retrieving patient information

[0637] 1. The user (staff member) speaks into the wearable device, "Please tell me the information for ID1234."

[0638] 2. The device converts the voice to text and sends the data "Please tell me the information about ID1234" to the server.

[0639] 3. The server retrieves the patient information corresponding to "ID1234" from the database.

[0640] 4. The server converts the acquired information into audio data and sends it to the device.

[0641] 5. The device outputs a voice message saying, "Yamada Taro has diabetes. His blood sugar levels have been stable recently. He needs to be careful about his diet."

[0642] Example 2: Updating patient information

[0643] 1. The user (staff member) speaks into the wearable device, "Record the recent status of ID1234."

[0644] 2. The device converts the voice to text and sends the data "Record the recent status of ID1234" to the server.

[0645] 3. The device will say, "Status has been checked. Please tell us the latest status."

[0646] 4. The user (staff member) speaks, "My blood sugar level was high yesterday."

[0647] 5. The device converts the voice into text and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[0648] 6. The server updates the relevant patient information in the database.

[0649] As described above, the system of the present invention enables nursing home staff to provide prompt and appropriate care to each elderly person.

[0650] The processing flow will be explained below.

[0651] Patient information database registration

[0652] Step 1:

[0653] The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions into a dedicated form.

[0654] Step 2:

[0655] The terminal converts the input information into a data package and transmits it to the server.

[0656] Step 3:

[0657] The server analyzes the received information and stores it in the database. For new patients, it creates a new record, and for existing patients, it updates the corresponding record.

[0658] Searching and retrieving patient information

[0659] Step 1:

[0660] The user (staff member) speaks the patient's identification information (e.g., ID1234) into the wearable device.

[0661] Step 2:

[0662] The device converts the voice input into text data in real time and sends the text data to the server.

[0663] Step 3:

[0664] The server searches the database based on the received text data and retrieves the relevant patient information.

[0665] Step 4:

[0666] The server converts the acquired patient information into text or voice data and sends it to the terminal.

[0667] Step 5:

[0668] The patient information received by the terminal is output as voice or displayed on the screen.

[0669] Patient Information Updates

[0670] Step 1:

[0671] The user (staff member) speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID 1234").

[0672] Step 2:

[0673] The device converts the voice into text data and sends it to the server.

[0674] Step 3:

[0675] The device will output a voice message saying, "Status has been checked. Please tell us the latest status."

[0676] Step 4:

[0677] The user (staff member) inputs by voice, "My blood sugar level was high yesterday."

[0678] Step 5:

[0679] The device converts the voice into text data, creating text data such as "yesterday's blood sugar level was high."

[0680] Step 6:

[0681] The terminal transmits the text data to the server.

[0682] Step 7:

[0683] The server updates the relevant patient information in the database.

[0684] Example 1

[0685] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0686] Conventional patient information management systems in nursing care facilities require staff to manually input and manage patient information, placing a heavy burden on staff and taking time to search for and update information.In addition, since information cannot be obtained or updated in real time via voice input, efficient information management is not possible in emergencies or when staff are busy.

[0687] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0688] In this invention, the server includes means for registering patient information in a database, means for acquiring patient identification information by voice recognition, means for acquiring corresponding patient information from the database based on the patient identification information, means for converting voice-input patient condition information into text and updating the corresponding patient information in the database, and means for processing the voice recognition results in real time and transmitting the acquired patient information to the wearable device, thereby enabling staff to efficiently manage patient information using voice input and acquire and update information in real time.

[0689] "Means for registering patient information in a database" refers to a system for storing information such as the patient's name, ID, medical history, recent condition, and precautions in a database.

[0690] The "means for acquiring patient identification information by voice recognition" is a mechanism for recognizing patient identification information input by voice, converting it into text data, and acquiring it.

[0691] The "means for obtaining relevant patient information from a database based on patient identification information" refers to a mechanism for searching a database based on identification information obtained by voice recognition and obtaining corresponding patient information.

[0692] "Means for outputting acquired patient information by voice or displaying it on a screen" refers to a system for conveying patient information acquired from a database to staff through voice or a display.

[0693] "Means for converting voice-input patient condition information into text and updating the corresponding patient information in the database" refers to a system in which staff converts new patient condition information entered by voice into text and updates the corresponding information in the database based on that information.

[0694] "Means for processing voice recognition results in real time and transmitting acquired patient information to a wearable device" refers to a mechanism for instantly processing the results obtained by voice recognition and transmitting acquired patient information based on the results to a wearable device.

[0695] The present invention relates to an information sharing system that enables staff in a nursing home to efficiently provide appropriate care to individual patients. The operation of the system of the present invention will be specifically described below.

[0696] System Overview

[0697] The system consists of the following elements:

[0698] 1. Database: Stores information such as patient name, ID, medical history, recent condition, and notes.

[0699] 2. Speech recognition device: Converts patient identification information entered by staff into text data using the Google Cloud Speech-to-Text API.

[0700] 3. Information acquisition device: Extracts relevant patient information from the database based on the converted text data.

[0701] 4. Audio output and display devices: Provide acquired patient information to staff. Use Web Speech API for audio output and a standard tablet or smartphone for display.

[0702] 5. Updater: Converts new patient status information entered by voice into text and updates the database.

[0703] Program processing

[0704] Patient information registration

[0705] 1. The user (administrator) uses a dedicated input form to enter information such as the patient's name, ID, medical history, recent condition, and precautions.

[0706] 2. The terminal packages the entered information and sends it to the server.

[0707] 3. The server stores the received information in the database, creating a new record for a new patient and updating the existing record for an existing patient.

[0708] Searching and retrieving patient information

[0709] 1. The user (staff member) speaks the patient's identification information into the wearable device.

[0710] 2. The device converts the voice input into text data in real time and sends the text data to the server.

[0711] 3. The server searches the database based on the received text data and retrieves the relevant patient information.

[0712] 4. The server sends the acquired patient information to the terminal in voice or text format.

[0713] 5. The terminal outputs the received patient information by voice or displays it on the display.

[0714] Patient Information Updates

[0715] 1. The user (staff member) speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID 1234").

[0716] 2. The device converts the voice into text data and sends it to the server.

[0717] 3. The server updates the relevant patient information in the database based on the text data received.

[0718] Specific examples

[0719] Example 1: Retrieving patient information

[0720] 1. The user (staff member) speaks into the wearable device, "Please tell me the information for ID1234."

[0721] 2. The device converts the voice to text and sends the data "Please tell me the information about ID1234" to the server.

[0722] 3. The server retrieves the patient information corresponding to "ID1234" from the database.

[0723] 4. The server converts the acquired information into audio data and sends it to the device.

[0724] 5. The device outputs a voice message saying, "Yamada Taro has diabetes. His blood sugar levels have been stable recently. He needs to be careful about his diet."

[0725] Example 2: Updating patient information

[0726] 1. The user (staff member) speaks into the wearable device, "Record the recent status of ID1234."

[0727] 2. The device converts the voice to text and sends the data "Record the recent status of ID1234" to the server.

[0728] 3. The device will say, "Status has been checked. Please tell us the latest status."

[0729] 4. The user (staff member) speaks, "My blood sugar level was high yesterday."

[0730] 5. The device converts the voice into text and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[0731] 6. The server updates the relevant patient information in the database.

[0732] As described above, the system of the present invention enables nursing home staff to obtain and update patient information in real time, enabling them to provide prompt and appropriate care.

[0733] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0734] Patient information registration

[0735] Step 1:

[0736] The user enters patient information using an input form. The information to be entered includes the patient's name, ID, medical history, recent condition, and precautions. Specifically, the user enters each item into the form on the web browser and clicks the submit button. The input data at this point is the string data for each item entered by the user.

[0737] Step 2:

[0738] The device packages the input information and sends it to the server. Specifically, JavaScript running on the device converts the input data into JSON format and sends it to the server via an HTTP POST request. The input is patient information in JSON format, and the output is an HTTP request.

[0739] Step 3:

[0740] The server stores the received information in the database. During this process, the server parses the HTTP POST request and extracts the JSON data. For new patients, it creates a new record using the SQL INSERT statement, and for existing patients, it updates the record using the UPDATE statement. The input is JSON-formatted data, and the output is the patient information stored in the database.

[0741] Searching and retrieving patient information

[0742] Step 1:

[0743] The user speaks the patient's identification information into the wearable device. Specifically, the user presses the voice input button and says, "Tell me the information about ID1234." The input data at this point is voice data.

[0744] Step 2:

[0745] The device converts voice input into text data in real time and sends that text data to the server. Specifically, a speech recognition engine (e.g., Google Cloud Speech-to-Text API) analyzes the voice data and generates text data such as "Please tell me the information about ID1234." This text data is then sent to the server as an HTTP request. The input is voice data, and the output is text data and an HTTP request.

[0746] Step 3:

[0747] The server searches the database based on the received text data and retrieves the corresponding patient information. Specifically, the server analyzes the text data and retrieves the patient information from the database using an SQL SELECT statement. The input is text data, and the output is the corresponding patient information.

[0748] Step 4:

[0749] The server acquires patient information and sends it to the terminal in voice or text format. The server converts the acquired information into JSON format and sends it to the terminal as an HTTP response. The input is patient information and the output is an HTTP response.

[0750] Step 5:

[0751] The terminal outputs the received patient information by voice or displays it on the display. Specifically, it uses a voice output library (e.g., SpeechSynthesis of the Web Speech API) to convey the information to the user by voice, or displays it as text on the display. The input is patient information in JSON format, and the output is voice data or display.

[0752] Patient Information Updates

[0753] Step 1:

[0754] The user speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID1234"). The input data is voice data.

[0755] Step 2:

[0756] The device converts the voice into text data and sends it to the server. Specifically, the voice recognition engine analyzes the voice data, generates text data such as "Record the recent status of ID1234," and sends it to the server as an HTTP request. The input is voice data, and the output is text data and an HTTP request.

[0757] Step 3:

[0758] The device will output a voice message saying, "The status has been checked. Please tell us the latest status." Specifically, a voice output library is used to play a voice message prompting the user for confirmation. The input is the text data mentioned earlier, and the output is a voice confirmation.

[0759] Step 4:

[0760] The user inputs by voice, "My blood sugar level was high yesterday." The input data is voice data.

[0761] Step 5:

[0762] The device again converts the voice into text data and sends it to the server. Specifically, the voice recognition engine generates text data such as "yesterday's blood sugar level was high" and sends it to the server as an HTTP request. The input is voice data, and the output is text data and an HTTP request.

[0763] Step 6:

[0764] The server updates the corresponding patient information in the database. Specifically, it uses the SQL UPDATE statement to update the corresponding database record based on the text data received by the server. The input is the text data, and the output is the updated database record.

[0765] (Application example 1)

[0766] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0767] In traditional nursing homes and factories, staff were required to efficiently obtain information about patients and equipment and respond appropriately. However, registering and updating information was time-consuming and there was a high possibility of input errors. In addition, it was difficult to obtain the necessary information in real time, which sometimes resulted in delayed or incorrect responses. This led to issues such as reduced operational efficiency at nursing homes and factories and inadequate management of patients and equipment.

[0768] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0769] In this invention, the server includes a means for registering patient information in a database, a means for acquiring patient identification information through voice recognition, a means for registering the operating status of multiple devices in the database, and a means for acquiring device identification information through voice recognition. This allows staff to quickly acquire patient and device information and respond appropriately. Furthermore, by enabling the registration and updating of information through voice recognition, it is possible to prevent input errors and improve operational efficiency.

[0770] "Patient information" refers to information such as the patient's name, ID, medical history, recent condition, and precautions.

[0771] A "database" is a collection of information designed to efficiently store specific information and to make it easy to search, update, and delete.

[0772] "Speech recognition" is a technology that converts speech into text data, analyzes speech input, and outputs a corresponding string of characters.

[0773] "Identification information" refers to information that uniquely identifies a specific individual or device, and includes an ID, name, etc.

[0774] "Audio output" is a technology that outputs audio data in a format that humans can hear through devices such as speakers.

[0775] "Display" is a technology for displaying information such as text and images on the screen of a display device.

[0776] "Device information" refers to information including the operating status of machines and robots, error information, maintenance information, and the like.

[0777] A "wearable device" is a computing device that can be worn by a user, including smart glasses and head-mounted displays.

[0778] The present invention relates to an information sharing system that enables staff in nursing homes and other care facilities, as well as in factories, to efficiently provide appropriate care and maintenance for individual patients and devices. The operation of the system of the present invention will be specifically described below.

[0779] System Overview

[0780] The system of the present invention comprises the following elements:

[0781] 1. Database

[0782] Stores patient and device information. Patient information includes name, ID, medical history, recent condition, and precautions, while device information includes operating status, error information, and maintenance information.

[0783] 2. Voice Recognition Device

[0784] The system converts patient and device identification information entered by staff via voice into text data using Python and SpeechRecognition (Google Speech-to-Text API).

[0785] 3. Information acquisition device

[0786] Based on the converted text data, the relevant patient and device information is extracted from the database using SQLite3.

[0787] 4. Audio output devices and display devices

[0788] The acquired patient and device information is provided to staff using speakers for audio output and tablets or smart glasses for display.

[0789] 5. Update device

[0790] Converts new spoken patient and device status information into text and updates the database. Uses Python, SpeechRecognition, and SQLite3.

[0791] Program processing

[0792] Patient information registration

[0793] The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions using a dedicated input form, and the terminal packages the entered information and sends it to the server. The server stores the received information in a database. At this time, a new record is created for new patients, and the corresponding record is updated for existing patients.

[0794] Registering device information

[0795] The user (administrator) enters information such as the device ID, operating status, error information, and maintenance information using a dedicated input form, and the terminal packages the entered information and sends it to the server. The server stores the received information in a database. At this time, a new record is created for new devices, and the corresponding record is updated for existing devices.

[0796] Search and retrieve patient and device information

[0797] The user (staff member) speaks into the wearable terminal to input the patient or device identification information, which the terminal converts into text data and sends to the server. The server searches the database based on the received text data and obtains the relevant patient and device information. The server then sends the obtained information to the terminal in voice or text format, and the terminal outputs the received information as voice or displays it on a display.

[0798] Specific examples

[0799] Example 1: Retrieving patient information

[0800] The user (staff member) speaks into the wearable device, saying, "Tell me the information about patient ID 1234." The device converts the speech into text and sends the data "Tell me the information about patient ID 1234" to the server. The server retrieves the patient information corresponding to "patient ID 1234" from the database. The server converts the retrieved information into voice data and sends it to the device. The device outputs a voice message saying, "Patient Name has diabetes. His blood sugar level has been stable recently. He needs to be careful about his diet."

[0801] Example 2: Updating Device Information

[0802] The user (staff member) speaks to the wearable device, saying, "The status of device ID 002 will be updated. The new status is an error, and the details are motor failure." The device converts the speech into text and sends the data "The status of device ID 002 will be updated. The new status is an error, and the details are motor failure" to the server. The server updates the relevant device information in the database, and the device outputs a voice message saying, "The status has been updated."

[0803] Prompt Sentence Examples

[0804] 1. "Please check the status of the following machine: Machine ID: 12345"

[0805] 2. "Please update the status of machine ID: 12345. New status: Under maintenance, Details: Part replaced"

[0806] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0807] Step 1:

[0808] The user inputs the patient or device identification information (e.g., patient ID or device ID) by voice. The input voice data is picked up by the microphone. The picked up voice data is sent by the terminal to the voice recognition engine (SpeechRecognition). The voice recognition engine converts the voice data into text data and outputs the text data to the terminal.

[0809] Input: Voice data (patient or device identification information)

[0810] Output: Text data (converted identification information)

[0811] Step 2:

[0812] The device receives the text data, packages it, and sends it to the server. The device converts the text data to JSON format and sends it as an HTTP request to the server. The server receives the HTTP request and prepares a database query.

[0813] Input: Text data of identification information

[0814] Output: HTTP request sent to the server

[0815] Step 3:

[0816] The server searches the database (SQLite3) based on the received text data and retrieves the corresponding patient or device information. The server executes an SQL query and retrieves the corresponding records. The retrieved information is converted to JSON format on the server and sent to the device as an HTTP response.

[0817] Input: The identification information sent to the server

[0818] Output: JSON data of the relevant patient or device information

[0819] Step 4:

[0820] The device analyzes the JSON data received from the server and provides information to the user through a voice output device or display device. The device parses the JSON data and converts it into a format for a speech synthesis engine or display. The speech synthesis engine converts the text data into audio data and plays it over the speaker. The display device displays the formatted text.

[0821] Input: JSON data from the server

[0822] Output: Audio output or display

[0823] Step 5:

[0824] When the user speaks a new patient or device status, the device again converts the voice to text and sends it to the server. The device then uses a speech recognition engine to convert the voice data to text, generates JSON data containing the updated information, and sends it to the server.

[0825] Input: Audio data (new state information)

[0826] Output: HTTP request sent to the server

[0827] Step 6:

[0828] The server updates the corresponding patient or device record in the database based on the received text data. The server executes the SQL UPDATE statement to update the corresponding record in the database with the new state. After the update is complete, the server returns a notification of update completion to the terminal.

[0829] Input: Text data of the update contents

[0830] Output: Notification of update completion

[0831] Step 7:

[0832] The device receives the update completion notification from the server and notifies the user by voice or display that the update is complete. A voice synthesis engine is used to play a message indicating that the update is complete, or a message is displayed on the screen.

[0833] Input: Notification of update completion from the server

[0834] Output: Update completion notification via audio output or display

[0835] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0836] This invention relates to an information sharing system that enables staff in nursing homes and other care facilities to efficiently provide appropriate care to each elderly person. By combining this system with an emotion engine that recognizes the user's emotions, it becomes possible to provide even more accurate care.

[0837] System Overview

[0838] The system of the present invention comprises the following elements:

[0839] 1. Database: Stores information such as patient name, ID, medical history, recent condition, and notes.

[0840] 2. Voice recognition device: Converts patient identification information entered by staff into text data.

[0841] 3. Information acquisition device: Extracts relevant patient information from the database based on the converted text data.

[0842] 4. Audio output and display devices: Provide retrieved patient information to staff.

[0843] 5. Updater: Converts new patient status information entered by voice into text and updates the database.

[0844] 6. Emotion Engine: Recognizes user emotions in real time and adjusts the information provided as needed.

[0845] Program processing

[0846] Patient information registration

[0847] 1. The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions into a dedicated form.

[0848] 2. The terminal packages the input information into a data package and sends it to the server.

[0849] 3. The server analyzes the received information and stores it in the database, creating a new record for a new patient and updating the existing record for an existing patient.

[0850] Searching and retrieving patient information

[0851] 1. The user (staff member) speaks the patient's identification information (e.g., ID1234) into the wearable device.

[0852] 2. The device converts the voice input into text data in real time and sends the text data to the server.

[0853] 3. The server searches the database based on the received text data and retrieves the relevant patient information.

[0854] 4. The server converts the acquired patient information into text or voice data and sends it to the terminal.

[0855] 5. The terminal outputs the received patient information by voice or displays it on the display.

[0856] Patient Information Updates

[0857] 1. The user (staff member) speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID 1234").

[0858] 2. The device converts the voice into text data and sends it to the server.

[0859] 3. The device will say, "Status has been checked. Please tell us the latest status."

[0860] 4. The user (staff member) speaks, "My blood sugar level was high yesterday."

[0861] 5. The device converts the voice into text data and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[0862] 6. The server updates the relevant patient information in the database.

[0863] Recognizing and modulating emotional information

[0864] 1. The device captures the user's (staff member's) voice and facial expressions in real time and sends them to the emotion engine.

[0865] 2. The emotion engine recognizes the user's (staff's) emotions from the received data and captures them as text or metadata.

[0866] 3. The emotion engine adjusts the content and expression of the patient information sent from the server based on the emotion.

[0867] 4. The server sends the adjusted patient information to the terminal and provides it to the staff in an appropriate format.

[0868] Specific examples

[0869] Example 1: Retrieving patient information

[0870] 1. The user (staff member) speaks into the wearable device, "Please tell me the information for ID1234."

[0871] 2. The device converts the voice to text and sends the data "Please tell me the information about ID1234" to the server.

[0872] 3. The server retrieves the patient information corresponding to "ID1234" from the database.

[0873] 4. The emotion engine analyzes the user's (staff's) current emotions and adjusts the way information is presented as needed (e.g., if the user is feeling stressed, it summarizes the output and presents it accordingly).

[0874] 5. The server converts the adjusted information into audio data and sends it to the device.

[0875] 6. The device outputs a voice message saying, "Yamada Taro has diabetes. His blood sugar levels have been stable recently. He needs to be careful about his diet."

[0876] Example 2: Updating patient information

[0877] 1. The user (staff member) speaks into the wearable device, "Record the recent status of ID1234."

[0878] 2. The device converts the voice to text and sends the data "Record the recent status of ID1234" to the server.

[0879] 3. The device will say, "Status has been checked. Please tell us the latest status."

[0880] 4. The user (staff member) speaks, "My blood sugar level was high yesterday."

[0881] 5. The device converts the voice into text and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[0882] 6. The sentiment engine analyzes the user's emotions and asks additional questions to ensure the input is accurate (e.g., "Was it really expensive?").

[0883] 7. The server updates the patient information in the database.

[0884] As described above, the system of the present invention enables nursing home staff to provide prompt and appropriate care to each elderly person, and by incorporating the user's emotions, the accuracy of information and the quality of care can be improved.

[0885] The processing flow will be explained below.

[0886] Patient information database registration

[0887] Step 1:

[0888] The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions into a dedicated form.

[0889] Step 2:

[0890] The terminal converts the input information into a data package and transmits it to the server.

[0891] Step 3:

[0892] The server analyzes the received information and stores it in the database. For new patients, it creates a new record, and for existing patients, it updates the corresponding record.

[0893] Searching and retrieving patient information

[0894] Step 1:

[0895] The user (staff member) speaks the patient's identification information (e.g., ID1234) into the wearable device.

[0896] Step 2:

[0897] The device converts the voice input into text data in real time and sends the text data to the server.

[0898] Step 3:

[0899] The server searches the database based on the received text data and retrieves the relevant patient information.

[0900] Step 4:

[0901] The device captures the user's (staff member's) voice and facial expressions and sends them to the emotion engine.

[0902] Step 5:

[0903] The emotion engine recognizes the user's (staff's) emotions from the received data and generates emotion information.

[0904] Step 6:

[0905] The server adjusts the acquired patient information based on the emotion information and summarizes the information as necessary.

[0906] Step 7:

[0907] The server converts the adjusted patient information into voice or text data and transmits it to the terminal.

[0908] Step 8:

[0909] The patient information received by the terminal is output as voice or displayed on the screen.

[0910] Patient Information Updates

[0911] Step 1:

[0912] The user (staff member) speaks into the wearable device, "Record the recent status of ID1234."

[0913] Step 2:

[0914] The device converts the voice into text data and sends it to the server.

[0915] Step 3:

[0916] The device will output a voice message saying, "Status has been checked. Please tell us the latest status."

[0917] Step 4:

[0918] The user (staff member) inputs by voice, "My blood sugar level was high yesterday."

[0919] Step 5:

[0920] The device converts the voice into text data, generating text data such as "yesterday's blood sugar level was high."

[0921] Step 6:

[0922] The terminal transmits the text data to the server.

[0923] Step 7:

[0924] The server analyzes the text data and updates the relevant patient information in the database.

[0925] Step 8:

[0926] The device captures the user's (staff member's) voice and facial expressions and sends them to the emotion engine.

[0927] Step 9:

[0928] The emotion engine recognizes the user's (staff's) emotions from the received data and adds questions to confirm the accuracy of the input.

[0929] Step 10:

[0930] The terminal transmits the re-entered information to the server.

[0931] Step 11:

[0932] The server will then finalize the patient information in the database and update it if necessary.

[0933] Example 2

[0934] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0935] Conventional information sharing systems in nursing homes mainly involve managing patient information manually, which often lacks efficiency and accuracy. Furthermore, information is provided without taking into account the emotions and stress levels of users (staff), making it impossible to improve the quality of care. It is necessary to provide a system that solves these problems and provides prompt and appropriate care to each elderly person.

[0936] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0937] In this invention, the server includes means for registering patient information in a database, means for acquiring patient identification information by voice recognition, means for acquiring corresponding patient information from the database based on the patient identification information, means for outputting the acquired patient information by voice or displaying it on a screen, means for adjusting the content and presentation method of information using an emotion engine that recognizes user emotions, means for converting voice-input patient condition information into text and updating the corresponding patient information in the database, means for processing the voice recognition results in real time and transmitting the acquired patient information to a wearable device, means for capturing the user's voice and facial expressions and transmitting them to the emotion engine, and means for adjusting the voice or text output based on the user's emotions. This allows information to be registered, acquired, and updated in the database accurately and efficiently, and by taking the user's emotions into consideration, it is possible to improve the quality of care.

[0938] The "means for registering patient information in a database" is a component that performs processing to store information such as the patient's name, ID, medical history, recent condition, and points to note in a database system.

[0939] The "means for acquiring patient identification information by voice recognition" is a mechanism for acquiring patient identification information input by voice as text data using voice recognition technology.

[0940] The "means for acquiring corresponding patient information from a database based on patient identification information" is a component that performs processing to search for and extract corresponding patient information from within a database using the acquired identification information.

[0941] "Means for outputting acquired patient information by voice or displaying it on a screen" refers to a mechanism for outputting or displaying information by voice or on a screen in order to convey searched or acquired patient information to the user.

[0942] The "emotion engine that recognizes user emotions" is a technology and software that analyzes and recognizes emotions from the user's voice and facial expressions in real time.

[0943] The "means for converting voice-inputted patient condition information into text and updating the corresponding patient information in the database" is a component that converts voice-inputted information into text and performs the process of updating the patient information stored in the database to the latest state.

[0944] The "means for processing voice recognition results in real time and transmitting the acquired patient information to a wearable terminal" is a mechanism for processing text data obtained by voice recognition in real time and transmitting the corresponding patient information to a wearable terminal.

[0945] The "means for capturing the user's voice and facial expression and transmitting it to the emotion engine" is a system for capturing the user's voice and facial expression and transmitting that data to the emotion engine.

[0946] "Means for adjusting voice or text output based on user emotions" refers to technology for adjusting the content and expression of the voice or text to be output in accordance with the user's emotions recognized by the emotion engine.

[0947] This invention relates to an information sharing system that enables staff in nursing homes and other care facilities to efficiently provide appropriate care to each elderly person. This system registers patient information in a database, acquires patient identification information using voice recognition technology, retrieves relevant patient information from the database based on that information, and outputs the acquired patient information as voice output or on a display. Furthermore, it differs from conventional systems in that it can recognize the user's emotions and adjust the content and presentation method of information based on those emotions.

[0948] The main components of the system are as follows:

[0949] Hardware and Software Components

[0950] 1. Database:

[0951] Store information such as the patient's name, ID, medical history, recent condition, and precautions. For example, use a relational database such as MySQL.

[0952] 2. Voice recognition device:

[0953] The patient's identification information is converted into text data by staff using the Google Cloud Speech-to-Text API.

[0954] 3. Information acquisition device:

[0955] Based on the converted text data, the relevant patient information is extracted from the database using a script (e.g., Python) that runs on the server.

[0956] 4. Audio output and display devices:

[0957] Use voice output devices and display terminals to provide retrieved patient information to staff. Examples include smartwatches and tablets.

[0958] 5. Update device:

[0959] New patient status information is spoken into the database and converted to text, using the Google Cloud Speech-to-Text API as well as a speech recognizer.

[0960] 6. Emotion Engine:

[0961] It recognizes the user's emotions in real time and adjusts the information provided as needed. As a concrete example, it uses the Microsoft Azure Emotion API.

[0962] Examples of concrete examples and prompts

[0963] 1. Obtaining patient information

[0964] The user (staff member) speaks into the wearable device, "Please tell me the information for ID1234."

[0965] The device converts the voice into text and sends the data "Please tell me the information about ID1234" to the server.

[0966] The server retrieves the patient information corresponding to "ID1234" from the database.

[0967] The emotion engine analyzes the user's (staff's) current emotions and adjusts the way information is presented as needed (e.g., if the user is feeling stressed, it summarizes the output and presents it accordingly).

[0968] The server converts the adjusted information into audio data and transmits it to the terminal.

[0969] The device outputs a voice message saying, "Yamada Taro has diabetes. His blood sugar levels have been stable recently. He needs to be careful about his diet."

[0970] 2. Updating patient information

[0971] The user (staff member) speaks into the wearable device, "Record the recent status of ID1234."

[0972] The device converts the voice into text and sends the data "Record the recent status of ID1234" to the server.

[0973] The device will output a voice message saying, "Status has been checked. Please tell us the latest status."

[0974] The user (staff member) inputs by voice, "My blood sugar level was high yesterday."

[0975] The device converts the voice into text and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[0976] The server updates the relevant patient information in the database.

[0977] By using these specific procedures and techniques, the present invention is designed to enable nursing home staff to provide more efficient and accurate care. As described above, the present invention enables the registration, acquisition, and updating of patient information quickly and accurately, and further improves the quality of care by taking into account the user's emotions.

[0978] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0979] Patient information registration

[0980] Step 1:

[0981] The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions into a dedicated form on the browser.

[0982] Specifically, the administrator enters information such as "Yamada Taro," "ID 1234," "has had diabetes for 10 years," "his blood sugar levels are stable these days," and "pay attention to dietary management."

[0983] Step 2:

[0984] The terminal converts the input information into a JSON format data package and sends it to the server.

[0985] Specifically, the device processes the input information into data such as "{"name":"Yamada Taro","ID":"ID1234","Medical history":"Diabetes for 10 years","Recent condition":"Stable blood sugar","Points to note":"Dietary management"}" and sends it to the server.

[0986] Step 3:

[0987] The server analyzes the received JSON data and stores it in the database.

[0988] Specifically, the server inserts a new record into the "patients" table or "updates" an existing record. Based on the JSON data as input, it generates a status indicating the data has been saved to the database as output.

[0989] Searching and retrieving patient information

[0990] Step 1:

[0991] The user (staff member) speaks into the wearable device to input the patient's identification information (e.g., "Please tell me the information for ID1234").

[0992] Specifically, the staff member speaks into the smartwatch, saying, "Tell me the information for ID1234."

[0993] Step 2:

[0994] The device converts the voice input into text data in real time and sends the text data to the server.

[0995] Specifically, the device converts the voice into text data such as "Tell me the information about ID1234" and sends it to the server.

[0996] Step 3:

[0997] The server queries the database based on the received text data and retrieves the corresponding patient information.

[0998] Specifically, the server executes the query "SELECT FROM patients WHERE ID='ID1234'." Based on the text data as input, corresponding patient information is obtained as output.

[0999] Step 4:

[1000] The server processes the acquired patient information into JSON format data and sends it to the terminal.

[1001] Specifically, the server generates the data "{"name":"Yamada Taro","ID":"ID1234","Medical history":"Diabetes for 10 years","Recent condition":"Stable blood sugar","Cautions":"Dietary management"}" and sends it to the terminal.

[1002] Step 5:

[1003] The patient information received by the terminal is output as voice or displayed on the screen.

[1004] Specifically, the device will output a message in voice or display on the screen saying, "Yamada Taro has diabetes. His blood sugar levels have been stable recently. He needs to be careful about his diet."

[1005] Patient Information Updates

[1006] Step 1:

[1007] The user (staff member) speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID 1234").

[1008] As for specific actions, the staff member said, "We will record the recent status of ID1234."

[1009] Step 2:

[1010] The device converts the voice into text data and sends it to the server.

[1011] Specifically, the terminal converts this into text data that reads "Record the recent status of ID1234" and sends it to the server.

[1012] Step 3:

[1013] The device will output a voice message saying, "Status has been checked. Please tell us the latest status."

[1014] Specifically, the device will announce in a voice message, "Status has been checked. Please tell us the latest status."

[1015] Step 4:

[1016] The user (staff member) inputs by voice, "My blood sugar level was high yesterday."

[1017] Specifically, the staff member says, "yesterday's blood sugar level was high."

[1018] Step 5:

[1019] The device converts the voice into text data and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[1020] Specifically, the device converts the text "yesterday's blood sugar level was high" and sends it to the server.

[1021] Step 6:

[1022] The server updates the relevant patient information in the database.

[1023] Specifically, the server executes the query "UPDATE patients SET recent status = 'high blood glucose level' WHERE ID = 'ID1234'". Based on the text data as input, the database is updated as output.

[1024] Recognizing and modulating emotional information

[1025] Step 1:

[1026] The device captures the user's (staff member's) voice and facial expressions in real time and sends them to the emotion engine.

[1027] Specifically, the device uses its built-in camera and microphone to capture the facial expressions and voice tone of the staff member.

[1028] Step 2:

[1029] The emotion engine recognizes the user's (staff's) emotions from the received data and captures them as text or metadata.

[1030] Specifically, the emotion engine generates emotion data such as "feeling stressed."

[1031] Step 3:

[1032] The emotion engine adjusts the content and expression of the patient information sent from the server based on the emotion.

[1033] Specifically, the emotion engine instructs the server to "summarize and present information to reduce stress."

[1034] Step 4:

[1035] The server sends the adjusted patient information to the terminal and provides it to the staff in an appropriate format.

[1036] Specifically, the server sends adjusted information to the terminal, such as "Yamada Taro has diabetes. He needs to be careful."

[1037] The above are the specific processing steps of the system, and the detailed operations and data processing / calculation contents at each step.

[1038] (Application example 2)

[1039] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1040] Nursing care facilities require an information sharing system that enables staff to provide prompt and appropriate care to patients. However, with existing systems, staff fatigue and stress can affect the quality of care, and the method of presenting information is not sufficiently optimized. Furthermore, there is a high risk of input errors when updating information via voice input, making information reliability an issue. This hinders the consistency and efficiency of the quality of care.

[1041] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1042] In this invention, the server includes means for registering patient information in a database, means for acquiring patient identification information by voice recognition, means for acquiring corresponding patient information from the database based on the patient identification information, means for recognizing staff emotions using an emotion engine, means for adjusting the method of providing patient information based on the recognized emotion, means for converting voice-input patient condition information into text and updating the corresponding patient information in the database, and means for processing the voice recognition results in real time and transmitting the acquired patient information to the wearable device. This makes it possible to optimize the method of providing information taking into account the emotions of staff and efficiently provide appropriate care to patients.

[1043] "Patient information" refers to data about individual patients, such as their name, ID, medical history, recent condition, and precautions.

[1044] A "database" is a collection of information that stores patient information in an organized manner and that can be quickly retrieved and updated as needed.

[1045] "Speech recognition" is a technology that converts speech into text data in real time.

[1046] "Patient identification information" refers to information for individually identifying a patient, and includes ID, name, etc.

[1047] "Audio output" is a means for providing acquired information to the user in audio form.

[1048] "Display" refers to a means for visually presenting acquired information to the user.

[1049] An "emotion engine" is a technology that recognizes a user's emotions by analyzing data such as voice and facial expressions.

[1050] A "wearable terminal" is an electronic device that can be worn by a user and is capable of receiving and outputting information.

[1051] "Real time" means that processing occurs the instant an event occurs.

[1052] MODE FOR CARRYING OUT THE INVENTION

[1053] System Configuration

[1054] The present invention is an information sharing system that comprises the following elements:

[1055] 1. Database:

[1056] Stores information such as patient name, ID, medical history, recent status, and precautions.

[1057] 2. Voice recognition device:

[1058] This device converts patient identification information entered by staff into text data. Specifically, smart glasses or wearable devices capture voice and convert it into text in real time.

[1059] 3. Information acquisition device:

[1060] Extract relevant patient information from the database. Search the database based on the text data converted by voice recognition to obtain the necessary patient information.

[1061] 4. Audio output and display devices:

[1062] This device outputs acquired patient information as voice or displays it on a display. It converts text data of patient information into voice data and provides it to the user via a wearable device.

[1063] 5. Update device:

[1064] This system converts new patient status information entered by voice into text and updates the database. Specifically, it has the function of sending the information entered by staff via voice to the server as text data and updating the corresponding record in the database.

[1065] 6. Emotion Engine:

[1066] This engine captures the user's emotions in real time from their voice and facial expressions, and adjusts the way information is presented as needed. This allows it to maintain the quality of care by adjusting the way information is presented if staff are under high stress or fatigue.

[1067] Example of a system

[1068] Example 1: Retrieving patient information

[1069] 1. The user (staff member) speaks into the smart glasses, "Please tell me the patient's ID number."

[1070] 2. The device converts the speech into text and sends it to the server.

[1071] 3. The server retrieves the relevant patient information from the database and analyzes the staff's emotions through the emotion engine.

[1072] 4. The emotion engine assesses whether stress is present and summarizes the information provided if necessary.

[1073] 5. The device outputs the information it has acquired in voice (e.g., "You are a diabetic patient. Your condition has recently stabilized.").

[1074] Example 2: Updating patient information

[1075] 1. The user (staff member) speaks into the smart glasses, "I would like to record the patient's condition."

[1076] 2. The device converts the speech into text and sends it to the server.

[1077] 3. The server retrieves the patient's information from the database and outputs a confirmation message.

[1078] 4. The user (staff member) then voice-inputs, "My blood pressure was high."

[1079] 5. The emotion engine analyzes the user's emotions and confirms the input.

[1080] 6. Update the relevant patient information in the database and notify the patient by voice message that "Information has been updated."

[1081] Hardware and software used

[1082] Hardware:

[1083] Smart glasses: capable of voice input and output

[1084] Microphone: Data acquisition for voice recognition

[1085] Camera: Data acquisition for emotion recognition

[1086] software:

[1087] Python: used as a programming language

[1088] SQLite3: Database Management

[1089] speech_recognition: Speech recognition engine

[1090] pyttsx3: Audio output engine

[1091] emotion_recognition (tentative): Emotion recognition engine

[1092] Prompt Sentence Examples

[1093] Emotion Engine Input:

[1094] python

[1095] emotion_engine.recognize("audio_data_file_path");

[1096] emotion_engine.recognize("video_data_file_path");

[1097] Speech recognition engine prompts:

[1098] python

[1099] recognizer.recognize_google(audio, language='ja-JP');

[1100] Using these technologies, the system provides powerful support for staff to provide patient care quickly and efficiently.

[1101] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1102] Program processing steps

[1103] Example 1: Retrieving patient information

[1104] Step 1:

[1105] The user (staff member) speaks into the smart glasses and says, "Please tell me the patient's ID number."

[1106] Input: Audio data

[1107] How it works: The microphone in the smart glasses captures audio.

[1108] Output: Captured audio data

[1109] Step 2:

[1110] The device calls the speech recognition engine (speech_recognition) to convert the voice data into text data.

[1111] Input: Audio data

[1112] How it works: A speech recognition engine converts voice data into text.

[1113] Output: Text data (e.g., "What is the patient's ID number?")

[1114] Step 3:

[1115] The terminal transmits the text data to the server, and the server retrieves the corresponding patient information from the database.

[1116] Input: Text data

[1117] Operation: Based on the text data, a database is queried to retrieve patient information.

[1118] Output: Patient information data

[1119] Step 4:

[1120] The server uses an emotion engine to recognize the emotions of staff and adjust the way information is presented.

[1121] Input: Patient information data, voice and facial expression data

[1122] How it works: The emotion engine analyzes voice and facial expression data to recognize the user's emotions, adjusting the patient summary and detailed information as needed.

[1123] Output: Reconciled patient information data

[1124] Step 5:

[1125] The terminal calls the voice output engine (pyttsx3) to output the adjusted patient information as voice.

[1126] Input: Adjusted patient information data

[1127] How it works: The voice output engine converts text data into voice data and outputs it from the speaker.

[1128] Output: Speech data (e.g., "I am a diabetic patient. My condition has stabilized recently.")

[1129] Example 2: Updating patient information

[1130] Step 1:

[1131] The user (staff member) speaks into the smart glasses and says, "I would like to record the patient's condition."

[1132] Input: Audio data

[1133] How it works: The microphone in the smart glasses captures audio.

[1134] Output: Captured audio data

[1135] Step 2:

[1136] The device calls the speech recognition engine (speech_recognition) to convert the voice data into text data.

[1137] Input: Audio data

[1138] How it works: A speech recognition engine converts voice data into text.

[1139] Output: Text data (e.g., "I want to record the patient's condition")

[1140] Step 3:

[1141] The terminal sends the text data to the server, which then retrieves the patient's information from the database and outputs a confirmation message.

[1142] Input: Text data

[1143] Behavior: Query database and generate confirmation message based on text data

[1144] Output: Text data of confirmation message

[1145] Step 4:

[1146] The device calls the voice output engine (pyttsx3) to output the confirmation message as voice.

[1147] Input: Text data of confirmation message

[1148] How it works: The voice output engine converts text data into voice data and outputs it from the speaker.

[1149] Output: Audio data (e.g., "We would like to record the patient's status. Do you wish to continue?")

[1150] Step 5:

[1151] The user (staff member) then inputs "My blood pressure was high" by voice.

[1152] Input: Audio data

[1153] How it works: The microphone in the smart glasses captures audio.

[1154] Output: Captured audio data

[1155] Step 6:

[1156] The terminal converts the voice data into text data and sends it to the server.

[1157] Input: Audio data

[1158] How it works: A speech recognition engine converts voice data into text.

[1159] Output: Text data (e.g., "My blood pressure was high")

[1160] Step 7:

[1161] The server uses an emotion engine to analyze the user's emotions and confirm the input content.

[1162] Input: Text data, voice and facial expression data

[1163] How it works: The emotion engine analyzes voice and facial expression data to recognize the user's emotions and generates a confirmation message if necessary.

[1164] Output: Text data of confirmation message (e.g., "Was your blood pressure really high?")

[1165] Step 8:

[1166] The relevant patient information in the database is updated and a voice message is generated saying, "The information has been updated."

[1167] Input: Verified text data

[1168] Behavior: Updates the corresponding record in the database and generates a confirmation message

[1169] Output: Text data of confirmation message

[1170] Step 9:

[1171] The device calls the voice output engine (pyttsx3) to output the confirmation message as voice.

[1172] Input: Text data of confirmation message

[1173] How it works: The voice output engine converts text data into voice data and outputs it from the speaker.

[1174] Output: Audio data (e.g. "Information has been updated")

[1175] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1176] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1177] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1178] [Third embodiment]

[1179] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1180] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1181] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1182] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1183] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1184] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1185] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1186] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1187] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1188] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1189] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1190] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1191] The present invention relates to an information sharing system that enables staff in nursing homes and other care facilities to efficiently provide appropriate care to individual elderly people. The following describes in detail how the system of the present invention operates.

[1192] System Overview

[1193] The system of the present invention comprises the following elements:

[1194] 1. Database: Stores information such as patient name, ID, medical history, recent condition, and notes.

[1195] 2. Voice recognition device: Converts patient identification information entered by staff into text data.

[1196] 3. Information acquisition device: Extracts relevant patient information from the database based on the converted text data.

[1197] 4. Audio output and display devices: Provide retrieved patient information to staff.

[1198] 5. Updater: Converts new patient status information entered by voice into text and updates the database.

[1199] Program processing

[1200] Patient information registration

[1201] 1. The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions using a dedicated input form.

[1202] 2. The terminal packages the entered information and sends it to the server.

[1203] 3. The server stores the received information in the database, creating a new record for a new patient and updating the existing record for an existing patient.

[1204] Searching and retrieving patient information

[1205] 1. The user (staff member) speaks the patient's identification information (e.g., ID1234) into the wearable device.

[1206] 2. The device converts the voice input into text data in real time and sends the text data to the server.

[1207] 3. The server searches the database based on the received text data and retrieves the relevant patient information.

[1208] 4. The server sends the acquired patient information to the terminal in voice or text format.

[1209] 5. The terminal outputs the received patient information by voice or displays it on the display.

[1210] Patient Information Updates

[1211] 1. The user (staff member) speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID 1234").

[1212] 2. The device converts the voice into text data and sends it to the server.

[1213] 3. The server updates the relevant patient information in the database based on the text data received.

[1214] Specific examples

[1215] Example 1: Retrieving patient information

[1216] 1. The user (staff member) speaks into the wearable device, "Please tell me the information for ID1234."

[1217] 2. The device converts the voice to text and sends the data "Please tell me the information about ID1234" to the server.

[1218] 3. The server retrieves the patient information corresponding to "ID1234" from the database.

[1219] 4. The server converts the acquired information into audio data and sends it to the device.

[1220] 5. The device outputs a voice message saying, "Yamada Taro has diabetes. His blood sugar levels have been stable recently. He needs to be careful about his diet."

[1221] Example 2: Updating patient information

[1222] 1. The user (staff member) speaks into the wearable device, "Record the recent status of ID1234."

[1223] 2. The device converts the voice to text and sends the data "Record the recent status of ID1234" to the server.

[1224] 3. The device will say, "Status has been checked. Please tell us the latest status."

[1225] 4. The user (staff member) speaks, "My blood sugar level was high yesterday."

[1226] 5. The device converts the voice into text and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[1227] 6. The server updates the relevant patient information in the database.

[1228] As described above, the system of the present invention enables nursing home staff to provide prompt and appropriate care to each elderly person.

[1229] The processing flow will be explained below.

[1230] Patient information database registration

[1231] Step 1:

[1232] The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions into a dedicated form.

[1233] Step 2:

[1234] The terminal converts the input information into a data package and transmits it to the server.

[1235] Step 3:

[1236] The server analyzes the received information and stores it in the database. For new patients, it creates a new record, and for existing patients, it updates the corresponding record.

[1237] Searching and retrieving patient information

[1238] Step 1:

[1239] The user (staff member) speaks the patient's identification information (e.g., ID1234) into the wearable device.

[1240] Step 2:

[1241] The device converts the voice input into text data in real time and sends the text data to the server.

[1242] Step 3:

[1243] The server searches the database based on the received text data and retrieves the relevant patient information.

[1244] Step 4:

[1245] The server converts the acquired patient information into text or voice data and sends it to the terminal.

[1246] Step 5:

[1247] The patient information received by the terminal is output as voice or displayed on the screen.

[1248] Patient Information Updates

[1249] Step 1:

[1250] The user (staff member) speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID 1234").

[1251] Step 2:

[1252] The device converts the voice into text data and sends it to the server.

[1253] Step 3:

[1254] The device will output a voice message saying, "Status has been checked. Please tell us the latest status."

[1255] Step 4:

[1256] The user (staff member) inputs by voice, "My blood sugar level was high yesterday."

[1257] Step 5:

[1258] The device converts the voice into text data, creating text data such as "yesterday's blood sugar level was high."

[1259] Step 6:

[1260] The terminal transmits the text data to the server.

[1261] Step 7:

[1262] The server updates the relevant patient information in the database.

[1263] Example 1

[1264] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1265] Conventional patient information management systems in nursing care facilities require staff to manually input and manage patient information, placing a heavy burden on staff and taking time to search for and update information.In addition, since information cannot be obtained or updated in real time via voice input, efficient information management is not possible in emergencies or when staff are busy.

[1266] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1267] In this invention, the server includes means for registering patient information in a database, means for acquiring patient identification information by voice recognition, means for acquiring corresponding patient information from the database based on the patient identification information, means for converting voice-input patient condition information into text and updating the corresponding patient information in the database, and means for processing the voice recognition results in real time and transmitting the acquired patient information to the wearable device, thereby enabling staff to efficiently manage patient information using voice input and acquire and update information in real time.

[1268] "Means for registering patient information in a database" refers to a system for storing information such as the patient's name, ID, medical history, recent condition, and precautions in a database.

[1269] The "means for acquiring patient identification information by voice recognition" is a mechanism for recognizing patient identification information input by voice, converting it into text data, and acquiring it.

[1270] The "means for obtaining relevant patient information from a database based on patient identification information" refers to a mechanism for searching a database based on identification information obtained by voice recognition and obtaining corresponding patient information.

[1271] "Means for outputting acquired patient information by voice or displaying it on a screen" refers to a system for conveying patient information acquired from a database to staff through voice or a display.

[1272] "Means for converting voice-input patient condition information into text and updating the corresponding patient information in the database" refers to a system in which staff converts new patient condition information entered by voice into text and updates the corresponding information in the database based on that information.

[1273] "Means for processing voice recognition results in real time and transmitting acquired patient information to a wearable device" refers to a mechanism for instantly processing the results obtained by voice recognition and transmitting acquired patient information based on the results to a wearable device.

[1274] The present invention relates to an information sharing system that enables staff in a nursing home to efficiently provide appropriate care to individual patients. The operation of the system of the present invention will be specifically described below.

[1275] System Overview

[1276] The system consists of the following elements:

[1277] 1. Database: Stores information such as patient name, ID, medical history, recent condition, and notes.

[1278] 2. Speech recognition device: Converts patient identification information entered by staff into text data using the Google Cloud Speech-to-Text API.

[1279] 3. Information acquisition device: Extracts relevant patient information from the database based on the converted text data.

[1280] 4. Audio output and display devices: Provide acquired patient information to staff. Use Web Speech API for audio output and a standard tablet or smartphone for display.

[1281] 5. Updater: Converts new patient status information entered by voice into text and updates the database.

[1282] Program processing

[1283] Patient information registration

[1284] 1. The user (administrator) uses a dedicated input form to enter information such as the patient's name, ID, medical history, recent condition, and precautions.

[1285] 2. The terminal packages the entered information and sends it to the server.

[1286] 3. The server stores the received information in the database, creating a new record for a new patient and updating the existing record for an existing patient.

[1287] Searching and retrieving patient information

[1288] 1. The user (staff member) speaks the patient's identification information into the wearable device.

[1289] 2. The device converts the voice input into text data in real time and sends the text data to the server.

[1290] 3. The server searches the database based on the received text data and retrieves the relevant patient information.

[1291] 4. The server sends the acquired patient information to the terminal in voice or text format.

[1292] 5. The terminal outputs the received patient information by voice or displays it on the display.

[1293] Patient Information Updates

[1294] 1. The user (staff member) speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID 1234").

[1295] 2. The device converts the voice into text data and sends it to the server.

[1296] 3. The server updates the relevant patient information in the database based on the text data received.

[1297] Specific examples

[1298] Example 1: Retrieving patient information

[1299] 1. The user (staff member) speaks into the wearable device, "Please tell me the information for ID1234."

[1300] 2. The device converts the voice to text and sends the data "Please tell me the information about ID1234" to the server.

[1301] 3. The server retrieves the patient information corresponding to "ID1234" from the database.

[1302] 4. The server converts the acquired information into audio data and sends it to the device.

[1303] 5. The device outputs a voice message saying, "Yamada Taro has diabetes. His blood sugar levels have been stable recently. He needs to be careful about his diet."

[1304] Example 2: Updating patient information

[1305] 1. The user (staff member) speaks into the wearable device, "Record the recent status of ID1234."

[1306] 2. The device converts the voice to text and sends the data "Record the recent status of ID1234" to the server.

[1307] 3. The device will say, "Status has been checked. Please tell us the latest status."

[1308] 4. The user (staff member) speaks, "My blood sugar level was high yesterday."

[1309] 5. The device converts the voice into text and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[1310] 6. The server updates the relevant patient information in the database.

[1311] As described above, the system of the present invention enables nursing home staff to obtain and update patient information in real time, enabling them to provide prompt and appropriate care.

[1312] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1313] Patient information registration

[1314] Step 1:

[1315] The user enters patient information using an input form. The information to be entered includes the patient's name, ID, medical history, recent condition, and precautions. Specifically, the user enters each item into the form on the web browser and clicks the submit button. The input data at this point is the string data for each item entered by the user.

[1316] Step 2:

[1317] The device packages the input information and sends it to the server. Specifically, JavaScript running on the device converts the input data into JSON format and sends it to the server via an HTTP POST request. The input is patient information in JSON format, and the output is an HTTP request.

[1318] Step 3:

[1319] The server stores the received information in the database. During this process, the server parses the HTTP POST request and extracts the JSON data. For new patients, it creates a new record using the SQL INSERT statement, and for existing patients, it updates the record using the UPDATE statement. The input is JSON-formatted data, and the output is the patient information stored in the database.

[1320] Searching and retrieving patient information

[1321] Step 1:

[1322] The user speaks the patient's identification information into the wearable device. Specifically, the user presses the voice input button and says, "Tell me the information about ID1234." The input data at this point is voice data.

[1323] Step 2:

[1324] The device converts voice input into text data in real time and sends that text data to the server. Specifically, a speech recognition engine (e.g., Google Cloud Speech-to-Text API) analyzes the voice data and generates text data such as "Please tell me the information about ID1234." This text data is then sent to the server as an HTTP request. The input is voice data, and the output is text data and an HTTP request.

[1325] Step 3:

[1326] The server searches the database based on the received text data and retrieves the corresponding patient information. Specifically, the server analyzes the text data and retrieves the patient information from the database using an SQL SELECT statement. The input is text data, and the output is the corresponding patient information.

[1327] Step 4:

[1328] The server acquires patient information and sends it to the terminal in voice or text format. The server converts the acquired information into JSON format and sends it to the terminal as an HTTP response. The input is patient information and the output is an HTTP response.

[1329] Step 5:

[1330] The terminal outputs the received patient information by voice or displays it on the display. Specifically, it uses a voice output library (e.g., SpeechSynthesis of the Web Speech API) to convey the information to the user by voice, or displays it as text on the display. The input is patient information in JSON format, and the output is voice data or display.

[1331] Patient Information Updates

[1332] Step 1:

[1333] The user speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID1234"). The input data is voice data.

[1334] Step 2:

[1335] The device converts the voice into text data and sends it to the server. Specifically, the voice recognition engine analyzes the voice data, generates text data such as "Record the recent status of ID1234," and sends it to the server as an HTTP request. The input is voice data, and the output is text data and an HTTP request.

[1336] Step 3:

[1337] The device will output a voice message saying, "The status has been checked. Please tell us the latest status." Specifically, a voice output library is used to play a voice message prompting the user for confirmation. The input is the text data mentioned earlier, and the output is a voice confirmation.

[1338] Step 4:

[1339] The user inputs by voice, "My blood sugar level was high yesterday." The input data is voice data.

[1340] Step 5:

[1341] The device again converts the voice into text data and sends it to the server. Specifically, the voice recognition engine generates text data such as "yesterday's blood sugar level was high" and sends it to the server as an HTTP request. The input is voice data, and the output is text data and an HTTP request.

[1342] Step 6:

[1343] The server updates the corresponding patient information in the database. Specifically, it uses the SQL UPDATE statement to update the corresponding database record based on the text data received by the server. The input is the text data, and the output is the updated database record.

[1344] (Application example 1)

[1345] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1346] In traditional nursing homes and factories, staff were required to efficiently obtain information about patients and equipment and respond appropriately. However, registering and updating information was time-consuming and there was a high possibility of input errors. In addition, it was difficult to obtain the necessary information in real time, which sometimes resulted in delayed or incorrect responses. This led to issues such as reduced operational efficiency at nursing homes and factories and inadequate management of patients and equipment.

[1347] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1348] In this invention, the server includes a means for registering patient information in a database, a means for acquiring patient identification information through voice recognition, a means for registering the operating status of multiple devices in the database, and a means for acquiring device identification information through voice recognition. This allows staff to quickly acquire patient and device information and respond appropriately. Furthermore, by enabling the registration and updating of information through voice recognition, it is possible to prevent input errors and improve operational efficiency.

[1349] "Patient information" refers to information such as the patient's name, ID, medical history, recent condition, and precautions.

[1350] A "database" is a collection of information designed to efficiently store specific information and to make it easy to search, update, and delete.

[1351] "Speech recognition" is a technology that converts speech into text data, analyzes speech input, and outputs a corresponding string of characters.

[1352] "Identification information" refers to information that uniquely identifies a specific individual or device, and includes an ID, name, etc.

[1353] "Audio output" is a technology that outputs audio data in a format that humans can hear through devices such as speakers.

[1354] "Display" is a technology for displaying information such as text and images on the screen of a display device.

[1355] "Device information" refers to information including the operating status of machines and robots, error information, maintenance information, and the like.

[1356] A "wearable device" is a computing device that can be worn by a user, including smart glasses and head-mounted displays.

[1357] The present invention relates to an information sharing system that enables staff in nursing homes and other care facilities, as well as in factories, to efficiently provide appropriate care and maintenance for individual patients and devices. The operation of the system of the present invention will be specifically described below.

[1358] System Overview

[1359] The system of the present invention comprises the following elements:

[1360] 1. Database

[1361] Stores patient and device information. Patient information includes name, ID, medical history, recent condition, and precautions, while device information includes operating status, error information, and maintenance information.

[1362] 2. Voice Recognition Device

[1363] The system converts patient and device identification information entered by staff via voice into text data using Python and SpeechRecognition (Google Speech-to-Text API).

[1364] 3. Information acquisition device

[1365] Based on the converted text data, the relevant patient and device information is extracted from the database using SQLite3.

[1366] 4. Audio output devices and display devices

[1367] The acquired patient and device information is provided to staff using speakers for audio output and tablets or smart glasses for display.

[1368] 5. Update device

[1369] Converts new spoken patient and device status information into text and updates the database. Uses Python, SpeechRecognition, and SQLite3.

[1370] Program processing

[1371] Patient information registration

[1372] The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions using a dedicated input form, and the terminal packages the entered information and sends it to the server. The server stores the received information in a database. At this time, a new record is created for new patients, and the corresponding record is updated for existing patients.

[1373] Registering device information

[1374] The user (administrator) enters information such as the device ID, operating status, error information, and maintenance information using a dedicated input form, and the terminal packages the entered information and sends it to the server. The server stores the received information in a database. At this time, a new record is created for new devices, and the corresponding record is updated for existing devices.

[1375] Search and retrieve patient and device information

[1376] The user (staff member) speaks into the wearable terminal to input the patient or device identification information, which the terminal converts into text data and sends to the server. The server searches the database based on the received text data and obtains the relevant patient and device information. The server then sends the obtained information to the terminal in voice or text format, and the terminal outputs the received information as voice or displays it on a display.

[1377] Specific examples

[1378] Example 1: Retrieving patient information

[1379] The user (staff member) speaks into the wearable device, saying, "Tell me the information about patient ID 1234." The device converts the speech into text and sends the data "Tell me the information about patient ID 1234" to the server. The server retrieves the patient information corresponding to "patient ID 1234" from the database. The server converts the retrieved information into voice data and sends it to the device. The device outputs a voice message saying, "Patient Name has diabetes. His blood sugar level has been stable recently. He needs to be careful about his diet."

[1380] Example 2: Updating Device Information

[1381] The user (staff member) speaks to the wearable device, saying, "The status of device ID 002 will be updated. The new status is an error, and the details are motor failure." The device converts the speech into text and sends the data "The status of device ID 002 will be updated. The new status is an error, and the details are motor failure" to the server. The server updates the relevant device information in the database, and the device outputs a voice message saying, "The status has been updated."

[1382] Prompt Sentence Examples

[1383] 1. "Please check the status of the following machine: Machine ID: 12345"

[1384] 2. "Please update the status of machine ID: 12345. New status: Under maintenance, Details: Part replaced"

[1385] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1386] Step 1:

[1387] The user inputs the patient or device identification information (e.g., patient ID or device ID) by voice. The input voice data is picked up by the microphone. The picked up voice data is sent by the terminal to the voice recognition engine (SpeechRecognition). The voice recognition engine converts the voice data into text data and outputs the text data to the terminal.

[1388] Input: Voice data (patient or device identification information)

[1389] Output: Text data (converted identification information)

[1390] Step 2:

[1391] The device receives the text data, packages it, and sends it to the server. The device converts the text data to JSON format and sends it as an HTTP request to the server. The server receives the HTTP request and prepares a database query.

[1392] Input: Text data of identification information

[1393] Output: HTTP request sent to the server

[1394] Step 3:

[1395] The server searches the database (SQLite3) based on the received text data and retrieves the corresponding patient or device information. The server executes an SQL query and retrieves the corresponding records. The retrieved information is converted to JSON format on the server and sent to the device as an HTTP response.

[1396] Input: The identification information sent to the server

[1397] Output: JSON data of the relevant patient or device information

[1398] Step 4:

[1399] The device analyzes the JSON data received from the server and provides information to the user through a voice output device or display device. The device parses the JSON data and converts it into a format for a speech synthesis engine or display. The speech synthesis engine converts the text data into audio data and plays it over the speaker. The display device displays the formatted text.

[1400] Input: JSON data from the server

[1401] Output: Audio output or display

[1402] Step 5:

[1403] When the user speaks a new patient or device status, the device again converts the voice to text and sends it to the server. The device then uses a speech recognition engine to convert the voice data to text, generates JSON data containing the updated information, and sends it to the server.

[1404] Input: Audio data (new state information)

[1405] Output: HTTP request sent to the server

[1406] Step 6:

[1407] The server updates the corresponding patient or device record in the database based on the received text data. The server executes the SQL UPDATE statement to update the corresponding record in the database with the new state. After the update is complete, the server returns a notification of update completion to the terminal.

[1408] Input: Text data of the update contents

[1409] Output: Notification of update completion

[1410] Step 7:

[1411] The device receives the update completion notification from the server and notifies the user by voice or display that the update is complete. A voice synthesis engine is used to play a message indicating that the update is complete, or a message is displayed on the screen.

[1412] Input: Notification of update completion from the server

[1413] Output: Update completion notification via audio output or display

[1414] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1415] This invention relates to an information sharing system that enables staff in nursing homes and other care facilities to efficiently provide appropriate care to each elderly person. By combining this system with an emotion engine that recognizes the user's emotions, it becomes possible to provide even more accurate care.

[1416] System Overview

[1417] The system of the present invention comprises the following elements:

[1418] 1. Database: Stores information such as patient name, ID, medical history, recent condition, and notes.

[1419] 2. Voice recognition device: Converts patient identification information entered by staff into text data.

[1420] 3. Information acquisition device: Extracts relevant patient information from the database based on the converted text data.

[1421] 4. Audio output and display devices: Provide retrieved patient information to staff.

[1422] 5. Updater: Converts new patient status information entered by voice into text and updates the database.

[1423] 6. Emotion Engine: Recognizes user emotions in real time and adjusts the information provided as needed.

[1424] Program processing

[1425] Patient information registration

[1426] 1. The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions into a dedicated form.

[1427] 2. The terminal packages the input information into a data package and sends it to the server.

[1428] 3. The server analyzes the received information and stores it in the database, creating a new record for a new patient and updating the existing record for an existing patient.

[1429] Searching and retrieving patient information

[1430] 1. The user (staff member) speaks the patient's identification information (e.g., ID1234) into the wearable device.

[1431] 2. The device converts the voice input into text data in real time and sends the text data to the server.

[1432] 3. The server searches the database based on the received text data and retrieves the relevant patient information.

[1433] 4. The server converts the acquired patient information into text or voice data and sends it to the terminal.

[1434] 5. The terminal outputs the received patient information by voice or displays it on the display.

[1435] Patient Information Updates

[1436] 1. The user (staff member) speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID 1234").

[1437] 2. The device converts the voice into text data and sends it to the server.

[1438] 3. The device will say, "Status has been checked. Please tell us the latest status."

[1439] 4. The user (staff member) speaks, "My blood sugar level was high yesterday."

[1440] 5. The device converts the voice into text data and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[1441] 6. The server updates the relevant patient information in the database.

[1442] Recognizing and modulating emotional information

[1443] 1. The device captures the user's (staff member's) voice and facial expressions in real time and sends them to the emotion engine.

[1444] 2. The emotion engine recognizes the user's (staff's) emotions from the received data and captures them as text or metadata.

[1445] 3. The emotion engine adjusts the content and expression of the patient information sent from the server based on the emotion.

[1446] 4. The server sends the adjusted patient information to the terminal and provides it to the staff in an appropriate format.

[1447] Specific examples

[1448] Example 1: Retrieving patient information

[1449] 1. The user (staff member) speaks into the wearable device, "Please tell me the information for ID1234."

[1450] 2. The device converts the voice to text and sends the data "Please tell me the information about ID1234" to the server.

[1451] 3. The server retrieves the patient information corresponding to "ID1234" from the database.

[1452] 4. The emotion engine analyzes the user's (staff's) current emotions and adjusts the way information is presented as needed (e.g., if the user is feeling stressed, it summarizes the output and presents it accordingly).

[1453] 5. The server converts the adjusted information into audio data and sends it to the device.

[1454] 6. The device outputs a voice message saying, "Yamada Taro has diabetes. His blood sugar levels have been stable recently. He needs to be careful about his diet."

[1455] Example 2: Updating patient information

[1456] 1. The user (staff member) speaks into the wearable device, "Record the recent status of ID1234."

[1457] 2. The device converts the voice to text and sends the data "Record the recent status of ID1234" to the server.

[1458] 3. The device will say, "Status has been checked. Please tell us the latest status."

[1459] 4. The user (staff member) speaks, "My blood sugar level was high yesterday."

[1460] 5. The device converts the voice into text and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[1461] 6. The sentiment engine analyzes the user's emotions and asks additional questions to ensure the input is accurate (e.g., "Was it really expensive?").

[1462] 7. The server updates the patient information in the database.

[1463] As described above, the system of the present invention enables nursing home staff to provide prompt and appropriate care to each elderly person, and by incorporating the user's emotions, the accuracy of information and the quality of care can be improved.

[1464] The processing flow will be explained below.

[1465] Patient information database registration

[1466] Step 1:

[1467] The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions into a dedicated form.

[1468] Step 2:

[1469] The terminal converts the input information into a data package and transmits it to the server.

[1470] Step 3:

[1471] The server analyzes the received information and stores it in the database. For new patients, it creates a new record, and for existing patients, it updates the corresponding record.

[1472] Searching and retrieving patient information

[1473] Step 1:

[1474] The user (staff member) speaks the patient's identification information (e.g., ID1234) into the wearable device.

[1475] Step 2:

[1476] The device converts the voice input into text data in real time and sends the text data to the server.

[1477] Step 3:

[1478] The server searches the database based on the received text data and retrieves the relevant patient information.

[1479] Step 4:

[1480] The device captures the user's (staff member's) voice and facial expressions and sends them to the emotion engine.

[1481] Step 5:

[1482] The emotion engine recognizes the user's (staff's) emotions from the received data and generates emotion information.

[1483] Step 6:

[1484] The server adjusts the acquired patient information based on the emotion information and summarizes the information as necessary.

[1485] Step 7:

[1486] The server converts the adjusted patient information into voice or text data and transmits it to the terminal.

[1487] Step 8:

[1488] The patient information received by the terminal is output as voice or displayed on the screen.

[1489] Patient Information Updates

[1490] Step 1:

[1491] The user (staff member) speaks into the wearable device, "Record the recent status of ID1234."

[1492] Step 2:

[1493] The device converts the voice into text data and sends it to the server.

[1494] Step 3:

[1495] The device will output a voice message saying, "Status has been checked. Please tell us the latest status."

[1496] Step 4:

[1497] The user (staff member) inputs by voice, "My blood sugar level was high yesterday."

[1498] Step 5:

[1499] The device converts the voice into text data, generating text data such as "yesterday's blood sugar level was high."

[1500] Step 6:

[1501] The terminal transmits the text data to the server.

[1502] Step 7:

[1503] The server analyzes the text data and updates the relevant patient information in the database.

[1504] Step 8:

[1505] The device captures the user's (staff member's) voice and facial expressions and sends them to the emotion engine.

[1506] Step 9:

[1507] The emotion engine recognizes the user's (staff's) emotions from the received data and adds questions to confirm the accuracy of the input.

[1508] Step 10:

[1509] The terminal transmits the re-entered information to the server.

[1510] Step 11:

[1511] The server will then finalize the patient information in the database and update it if necessary.

[1512] Example 2

[1513] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1514] Conventional information sharing systems in nursing homes mainly involve managing patient information manually, which often lacks efficiency and accuracy. Furthermore, information is provided without taking into account the emotions and stress levels of users (staff), making it impossible to improve the quality of care. It is necessary to provide a system that solves these problems and provides prompt and appropriate care to each elderly person.

[1515] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1516] In this invention, the server includes means for registering patient information in a database, means for acquiring patient identification information by voice recognition, means for acquiring corresponding patient information from the database based on the patient identification information, means for outputting the acquired patient information by voice or displaying it on a screen, means for adjusting the content and presentation method of information using an emotion engine that recognizes user emotions, means for converting voice-input patient condition information into text and updating the corresponding patient information in the database, means for processing the voice recognition results in real time and transmitting the acquired patient information to a wearable device, means for capturing the user's voice and facial expressions and transmitting them to the emotion engine, and means for adjusting the voice or text output based on the user's emotions. This allows information to be registered, acquired, and updated in the database accurately and efficiently, and by taking the user's emotions into consideration, it is possible to improve the quality of care.

[1517] The "means for registering patient information in a database" is a component that performs processing to store information such as the patient's name, ID, medical history, recent condition, and points to note in a database system.

[1518] The "means for acquiring patient identification information by voice recognition" is a mechanism for acquiring patient identification information input by voice as text data using voice recognition technology.

[1519] The "means for acquiring corresponding patient information from a database based on patient identification information" is a component that performs processing to search for and extract corresponding patient information from within a database using the acquired identification information.

[1520] "Means for outputting acquired patient information by voice or displaying it on a screen" refers to a mechanism for outputting or displaying information by voice or on a screen in order to convey searched or acquired patient information to the user.

[1521] The "emotion engine that recognizes user emotions" is a technology and software that analyzes and recognizes emotions from the user's voice and facial expressions in real time.

[1522] The "means for converting voice-inputted patient condition information into text and updating the corresponding patient information in the database" is a component that converts voice-inputted information into text and performs the process of updating the patient information stored in the database to the latest state.

[1523] The "means for processing voice recognition results in real time and transmitting the acquired patient information to a wearable terminal" is a mechanism for processing text data obtained by voice recognition in real time and transmitting the corresponding patient information to a wearable terminal.

[1524] The "means for capturing the user's voice and facial expression and transmitting it to the emotion engine" is a system for capturing the user's voice and facial expression and transmitting that data to the emotion engine.

[1525] "Means for adjusting voice or text output based on user emotions" refers to technology for adjusting the content and expression of the voice or text to be output in accordance with the user's emotions recognized by the emotion engine.

[1526] This invention relates to an information sharing system that enables staff in nursing homes and other care facilities to efficiently provide appropriate care to each elderly person. This system registers patient information in a database, acquires patient identification information using voice recognition technology, retrieves relevant patient information from the database based on that information, and outputs the acquired patient information as voice output or on a display. Furthermore, it differs from conventional systems in that it can recognize the user's emotions and adjust the content and presentation method of information based on those emotions.

[1527] The main components of the system are as follows:

[1528] Hardware and Software Components

[1529] 1. Database:

[1530] Store information such as the patient's name, ID, medical history, recent condition, and precautions. For example, use a relational database such as MySQL.

[1531] 2. Voice recognition device:

[1532] The patient's identification information is converted into text data by staff using the Google Cloud Speech-to-Text API.

[1533] 3. Information acquisition device:

[1534] Based on the converted text data, the relevant patient information is extracted from the database using a script (e.g., Python) that runs on the server.

[1535] 4. Audio output and display devices:

[1536] Use voice output devices and display terminals to provide retrieved patient information to staff. Examples include smartwatches and tablets.

[1537] 5. Update device:

[1538] New patient status information is spoken into the database and converted to text, using the Google Cloud Speech-to-Text API as well as a speech recognizer.

[1539] 6. Emotion Engine:

[1540] It recognizes the user's emotions in real time and adjusts the information provided as needed. As a concrete example, it uses the Microsoft Azure Emotion API.

[1541] Examples of concrete examples and prompts

[1542] 1. Obtaining patient information

[1543] The user (staff member) speaks into the wearable device, "Please tell me the information for ID1234."

[1544] The device converts the voice into text and sends the data "Please tell me the information about ID1234" to the server.

[1545] The server retrieves the patient information corresponding to "ID1234" from the database.

[1546] The emotion engine analyzes the user's (staff's) current emotions and adjusts the way information is presented as needed (e.g., if the user is feeling stressed, it summarizes the output and presents it accordingly).

[1547] The server converts the adjusted information into audio data and transmits it to the terminal.

[1548] The device outputs a voice message saying, "Yamada Taro has diabetes. His blood sugar levels have been stable recently. He needs to be careful about his diet."

[1549] 2. Updating patient information

[1550] The user (staff member) speaks into the wearable device, "Record the recent status of ID1234."

[1551] The device converts the voice into text and sends the data "Record the recent status of ID1234" to the server.

[1552] The device will output a voice message saying, "Status has been checked. Please tell us the latest status."

[1553] The user (staff member) inputs by voice, "My blood sugar level was high yesterday."

[1554] The device converts the voice into text and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[1555] The server updates the relevant patient information in the database.

[1556] By using these specific procedures and techniques, the present invention is designed to enable nursing home staff to provide more efficient and accurate care. As described above, the present invention enables the registration, acquisition, and updating of patient information quickly and accurately, and further improves the quality of care by taking into account the user's emotions.

[1557] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1558] Patient information registration

[1559] Step 1:

[1560] The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions into a dedicated form on the browser.

[1561] Specifically, the administrator enters information such as "Yamada Taro," "ID 1234," "has had diabetes for 10 years," "his blood sugar levels are stable these days," and "pay attention to dietary management."

[1562] Step 2:

[1563] The terminal converts the input information into a JSON format data package and sends it to the server.

[1564] Specifically, the device processes the input information into data such as "{"name":"Yamada Taro","ID":"ID1234","Medical history":"Diabetes for 10 years","Recent condition":"Stable blood sugar","Points to note":"Dietary management"}" and sends it to the server.

[1565] Step 3:

[1566] The server analyzes the received JSON data and stores it in the database.

[1567] Specifically, the server inserts a new record into the "patients" table or "updates" an existing record. Based on the JSON data as input, it generates a status indicating the data has been saved to the database as output.

[1568] Searching and retrieving patient information

[1569] Step 1:

[1570] The user (staff member) speaks into the wearable device to input the patient's identification information (e.g., "Please tell me the information for ID1234").

[1571] Specifically, the staff member speaks into the smartwatch, saying, "Tell me the information for ID1234."

[1572] Step 2:

[1573] The device converts the voice input into text data in real time and sends the text data to the server.

[1574] Specifically, the device converts the voice into text data such as "Tell me the information about ID1234" and sends it to the server.

[1575] Step 3:

[1576] The server queries the database based on the received text data and retrieves the corresponding patient information.

[1577] Specifically, the server executes the query "SELECT FROM patients WHERE ID='ID1234'." Based on the text data as input, corresponding patient information is obtained as output.

[1578] Step 4:

[1579] The server processes the acquired patient information into JSON format data and sends it to the terminal.

[1580] Specifically, the server generates the data "{"name":"Yamada Taro","ID":"ID1234","Medical history":"Diabetes for 10 years","Recent condition":"Stable blood sugar","Cautions":"Dietary management"}" and sends it to the terminal.

[1581] Step 5:

[1582] The patient information received by the terminal is output as voice or displayed on the screen.

[1583] Specifically, the device will output a message in voice or display on the screen saying, "Yamada Taro has diabetes. His blood sugar levels have been stable recently. He needs to be careful about his diet."

[1584] Patient Information Updates

[1585] Step 1:

[1586] The user (staff member) speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID 1234").

[1587] As for specific actions, the staff member said, "We will record the recent status of ID1234."

[1588] Step 2:

[1589] The device converts the voice into text data and sends it to the server.

[1590] Specifically, the terminal converts this into text data that reads "Record the recent status of ID1234" and sends it to the server.

[1591] Step 3:

[1592] The device will output a voice message saying, "Status has been checked. Please tell us the latest status."

[1593] Specifically, the device will announce in a voice message, "Status has been checked. Please tell us the latest status."

[1594] Step 4:

[1595] The user (staff member) inputs by voice, "My blood sugar level was high yesterday."

[1596] Specifically, the staff member says, "yesterday's blood sugar level was high."

[1597] Step 5:

[1598] The device converts the voice into text data and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[1599] Specifically, the device converts the text "yesterday's blood sugar level was high" and sends it to the server.

[1600] Step 6:

[1601] The server updates the relevant patient information in the database.

[1602] Specifically, the server executes the query "UPDATE patients SET recent status = 'high blood glucose level' WHERE ID = 'ID1234'". Based on the text data as input, the database is updated as output.

[1603] Recognizing and modulating emotional information

[1604] Step 1:

[1605] The device captures the user's (staff member's) voice and facial expressions in real time and sends them to the emotion engine.

[1606] Specifically, the device uses its built-in camera and microphone to capture the facial expressions and voice tone of the staff member.

[1607] Step 2:

[1608] The emotion engine recognizes the user's (staff's) emotions from the received data and captures them as text or metadata.

[1609] Specifically, the emotion engine generates emotion data such as "feeling stressed."

[1610] Step 3:

[1611] The emotion engine adjusts the content and expression of the patient information sent from the server based on the emotion.

[1612] Specifically, the emotion engine instructs the server to "summarize and present information to reduce stress."

[1613] Step 4:

[1614] The server sends the adjusted patient information to the terminal and provides it to the staff in an appropriate format.

[1615] Specifically, the server sends adjusted information to the terminal, such as "Yamada Taro has diabetes. He needs to be careful."

[1616] The above are the specific processing steps of the system, and the detailed operations and data processing / calculation contents at each step.

[1617] (Application example 2)

[1618] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1619] Nursing care facilities require an information sharing system that enables staff to provide prompt and appropriate care to patients. However, with existing systems, staff fatigue and stress can affect the quality of care, and the method of presenting information is not sufficiently optimized. Furthermore, there is a high risk of input errors when updating information via voice input, making information reliability an issue. This hinders the consistency and efficiency of the quality of care.

[1620] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1621] In this invention, the server includes means for registering patient information in a database, means for acquiring patient identification information by voice recognition, means for acquiring corresponding patient information from the database based on the patient identification information, means for recognizing staff emotions using an emotion engine, means for adjusting the method of providing patient information based on the recognized emotion, means for converting voice-input patient condition information into text and updating the corresponding patient information in the database, and means for processing the voice recognition results in real time and transmitting the acquired patient information to the wearable device. This makes it possible to optimize the method of providing information taking into account the emotions of staff and efficiently provide appropriate care to patients.

[1622] "Patient information" refers to data about individual patients, such as their name, ID, medical history, recent condition, and precautions.

[1623] A "database" is a collection of information that stores patient information in an organized manner and that can be quickly retrieved and updated as needed.

[1624] "Speech recognition" is a technology that converts speech into text data in real time.

[1625] "Patient identification information" refers to information for individually identifying a patient, and includes ID, name, etc.

[1626] "Audio output" is a means for providing acquired information to the user in audio form.

[1627] "Display" refers to a means for visually presenting acquired information to the user.

[1628] An "emotion engine" is a technology that recognizes a user's emotions by analyzing data such as voice and facial expressions.

[1629] A "wearable terminal" is an electronic device that can be worn by a user and is capable of receiving and outputting information.

[1630] "Real time" means that processing occurs the instant an event occurs.

[1631] MODE FOR CARRYING OUT THE INVENTION

[1632] System Configuration

[1633] The present invention is an information sharing system that comprises the following elements:

[1634] 1. Database:

[1635] Stores information such as patient name, ID, medical history, recent status, and precautions.

[1636] 2. Voice recognition device:

[1637] This device converts patient identification information entered by staff into text data. Specifically, smart glasses or wearable devices capture voice and convert it into text in real time.

[1638] 3. Information acquisition device:

[1639] Extract relevant patient information from the database. Search the database based on the text data converted by voice recognition to obtain the necessary patient information.

[1640] 4. Audio output and display devices:

[1641] This device outputs acquired patient information as voice or displays it on a display. It converts text data of patient information into voice data and provides it to the user via a wearable device.

[1642] 5. Update device:

[1643] This system converts new patient status information entered by voice into text and updates the database. Specifically, it has the function of sending the information entered by staff via voice to the server as text data and updating the corresponding record in the database.

[1644] 6. Emotion Engine:

[1645] This engine captures the user's emotions in real time from their voice and facial expressions, and adjusts the way information is presented as needed. This allows it to maintain the quality of care by adjusting the way information is presented if staff are under high stress or fatigue.

[1646] Example of a system

[1647] Example 1: Retrieving patient information

[1648] 1. The user (staff member) speaks into the smart glasses, "Please tell me the patient's ID number."

[1649] 2. The device converts the speech into text and sends it to the server.

[1650] 3. The server retrieves the relevant patient information from the database and analyzes the staff's emotions through the emotion engine.

[1651] 4. The emotion engine assesses whether stress is present and summarizes the information provided if necessary.

[1652] 5. The device outputs the information it has acquired in voice (e.g., "You are a diabetic patient. Your condition has recently stabilized.").

[1653] Example 2: Updating patient information

[1654] 1. The user (staff member) speaks into the smart glasses, "I would like to record the patient's condition."

[1655] 2. The device converts the speech into text and sends it to the server.

[1656] 3. The server retrieves the patient's information from the database and outputs a confirmation message.

[1657] 4. The user (staff member) then voice-inputs, "My blood pressure was high."

[1658] 5. The emotion engine analyzes the user's emotions and confirms the input.

[1659] 6. Update the relevant patient information in the database and notify the patient by voice message that "Information has been updated."

[1660] Hardware and software used

[1661] Hardware:

[1662] Smart glasses: capable of voice input and output

[1663] Microphone: Data acquisition for voice recognition

[1664] Camera: Data acquisition for emotion recognition

[1665] software:

[1666] Python: used as a programming language

[1667] SQLite3: Database Management

[1668] speech_recognition: Speech recognition engine

[1669] pyttsx3: Audio output engine

[1670] emotion_recognition (tentative): Emotion recognition engine

[1671] Prompt Sentence Examples

[1672] Emotion Engine Input:

[1673] python

[1674] emotion_engine.recognize("audio_data_file_path");

[1675] emotion_engine.recognize("video_data_file_path");

[1676] Speech recognition engine prompts:

[1677] python

[1678] recognizer.recognize_google(audio, language='ja-JP');

[1679] Using these technologies, the system provides powerful support for staff to provide patient care quickly and efficiently.

[1680] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1681] Program processing steps

[1682] Example 1: Retrieving patient information

[1683] Step 1:

[1684] The user (staff member) speaks into the smart glasses and says, "Please tell me the patient's ID number."

[1685] Input: Audio data

[1686] How it works: The microphone in the smart glasses captures audio.

[1687] Output: Captured audio data

[1688] Step 2:

[1689] The device calls the speech recognition engine (speech_recognition) to convert the voice data into text data.

[1690] Input: Audio data

[1691] How it works: A speech recognition engine converts voice data into text.

[1692] Output: Text data (e.g., "What is the patient's ID number?")

[1693] Step 3:

[1694] The terminal transmits the text data to the server, and the server retrieves the corresponding patient information from the database.

[1695] Input: Text data

[1696] Operation: Based on the text data, a database is queried to retrieve patient information.

[1697] Output: Patient information data

[1698] Step 4:

[1699] The server uses an emotion engine to recognize the emotions of staff and adjust the way information is presented.

[1700] Input: Patient information data, voice and facial expression data

[1701] How it works: The emotion engine analyzes voice and facial expression data to recognize the user's emotions, adjusting the patient summary and detailed information as needed.

[1702] Output: Reconciled patient information data

[1703] Step 5:

[1704] The terminal calls the voice output engine (pyttsx3) to output the adjusted patient information as voice.

[1705] Input: Adjusted patient information data

[1706] How it works: The voice output engine converts text data into voice data and outputs it from the speaker.

[1707] Output: Speech data (e.g., "I am a diabetic patient. My condition has stabilized recently.")

[1708] Example 2: Updating patient information

[1709] Step 1:

[1710] The user (staff member) speaks into the smart glasses and says, "I would like to record the patient's condition."

[1711] Input: Audio data

[1712] How it works: The microphone in the smart glasses captures audio.

[1713] Output: Captured audio data

[1714] Step 2:

[1715] The device calls the speech recognition engine (speech_recognition) to convert the voice data into text data.

[1716] Input: Audio data

[1717] How it works: A speech recognition engine converts voice data into text.

[1718] Output: Text data (e.g., "I want to record the patient's condition")

[1719] Step 3:

[1720] The terminal sends the text data to the server, which then retrieves the patient's information from the database and outputs a confirmation message.

[1721] Input: Text data

[1722] Behavior: Query database and generate confirmation message based on text data

[1723] Output: Text data of confirmation message

[1724] Step 4:

[1725] The device calls the voice output engine (pyttsx3) to output the confirmation message as voice.

[1726] Input: Text data of confirmation message

[1727] How it works: The voice output engine converts text data into voice data and outputs it from the speaker.

[1728] Output: Audio data (e.g., "We would like to record the patient's status. Do you wish to continue?")

[1729] Step 5:

[1730] The user (staff member) then inputs "My blood pressure was high" by voice.

[1731] Input: Audio data

[1732] How it works: The microphone in the smart glasses captures audio.

[1733] Output: Captured audio data

[1734] Step 6:

[1735] The terminal converts the voice data into text data and sends it to the server.

[1736] Input: Audio data

[1737] How it works: A speech recognition engine converts voice data into text.

[1738] Output: Text data (e.g., "My blood pressure was high")

[1739] Step 7:

[1740] The server uses an emotion engine to analyze the user's emotions and confirm the input content.

[1741] Input: Text data, voice and facial expression data

[1742] How it works: The emotion engine analyzes voice and facial expression data to recognize the user's emotions and generates a confirmation message if necessary.

[1743] Output: Text data of confirmation message (e.g., "Was your blood pressure really high?")

[1744] Step 8:

[1745] The relevant patient information in the database is updated and a voice message is generated saying, "The information has been updated."

[1746] Input: Verified text data

[1747] Behavior: Updates the corresponding record in the database and generates a confirmation message

[1748] Output: Text data of confirmation message

[1749] Step 9:

[1750] The device calls the voice output engine (pyttsx3) to output the confirmation message as voice.

[1751] Input: Text data of confirmation message

[1752] How it works: The voice output engine converts text data into voice data and outputs it from the speaker.

[1753] Output: Audio data (e.g. "Information has been updated")

[1754] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1755] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1756] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1757] [Fourth embodiment]

[1758] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1759] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1760] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1761] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1762] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1763] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1764] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1765] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1766] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1767] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1768] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1769] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1770] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1771] The present invention relates to an information sharing system that enables staff in nursing homes and other care facilities to efficiently provide appropriate care to individual elderly people. The following describes in detail how the system of the present invention operates.

[1772] System Overview

[1773] The system of the present invention comprises the following elements:

[1774] 1. Database: Stores information such as patient name, ID, medical history, recent condition, and notes.

[1775] 2. Voice recognition device: Converts patient identification information entered by staff into text data.

[1776] 3. Information acquisition device: Extracts relevant patient information from the database based on the converted text data.

[1777] 4. Audio output and display devices: Provide retrieved patient information to staff.

[1778] 5. Updater: Converts new patient status information entered by voice into text and updates the database.

[1779] Program processing

[1780] Patient information registration

[1781] 1. The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions using a dedicated input form.

[1782] 2. The terminal packages the entered information and sends it to the server.

[1783] 3. The server stores the received information in the database, creating a new record for a new patient and updating the existing record for an existing patient.

[1784] Searching and retrieving patient information

[1785] 1. The user (staff member) speaks the patient's identification information (e.g., ID1234) into the wearable device.

[1786] 2. The device converts the voice input into text data in real time and sends the text data to the server.

[1787] 3. The server searches the database based on the received text data and retrieves the relevant patient information.

[1788] 4. The server sends the acquired patient information to the terminal in voice or text format.

[1789] 5. The terminal outputs the received patient information by voice or displays it on the display.

[1790] Patient Information Updates

[1791] 1. The user (staff member) speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID 1234").

[1792] 2. The device converts the voice into text data and sends it to the server.

[1793] 3. The server updates the relevant patient information in the database based on the text data received.

[1794] Specific examples

[1795] Example 1: Retrieving patient information

[1796] 1. The user (staff member) speaks into the wearable device, "Please tell me the information for ID1234."

[1797] 2. The device converts the voice to text and sends the data "Please tell me the information about ID1234" to the server.

[1798] 3. The server retrieves the patient information corresponding to "ID1234" from the database.

[1799] 4. The server converts the acquired information into audio data and sends it to the device.

[1800] 5. The device outputs a voice message saying, "Yamada Taro has diabetes. His blood sugar levels have been stable recently. He needs to be careful about his diet."

[1801] Example 2: Updating patient information

[1802] 1. The user (staff member) speaks into the wearable device, "Record the recent status of ID1234."

[1803] 2. The device converts the voice to text and sends the data "Record the recent status of ID1234" to the server.

[1804] 3. The device will say, "Status has been checked. Please tell us the latest status."

[1805] 4. The user (staff member) speaks, "My blood sugar level was high yesterday."

[1806] 5. The device converts the voice into text and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[1807] 6. The server updates the relevant patient information in the database.

[1808] As described above, the system of the present invention enables nursing home staff to provide prompt and appropriate care to each elderly person.

[1809] The processing flow will be explained below.

[1810] Patient information database registration

[1811] Step 1:

[1812] The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions into a dedicated form.

[1813] Step 2:

[1814] The terminal converts the input information into a data package and transmits it to the server.

[1815] Step 3:

[1816] The server analyzes the received information and stores it in the database. For new patients, it creates a new record, and for existing patients, it updates the corresponding record.

[1817] Searching and retrieving patient information

[1818] Step 1:

[1819] The user (staff member) speaks the patient's identification information (e.g., ID1234) into the wearable device.

[1820] Step 2:

[1821] The device converts the voice input into text data in real time and sends the text data to the server.

[1822] Step 3:

[1823] The server searches the database based on the received text data and retrieves the relevant patient information.

[1824] Step 4:

[1825] The server converts the acquired patient information into text or voice data and sends it to the terminal.

[1826] Step 5:

[1827] The patient information received by the terminal is output as voice or displayed on the screen.

[1828] Patient Information Updates

[1829] Step 1:

[1830] The user (staff member) speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID 1234").

[1831] Step 2:

[1832] The device converts the voice into text data and sends it to the server.

[1833] Step 3:

[1834] The device will output a voice message saying, "Status has been checked. Please tell us the latest status."

[1835] Step 4:

[1836] The user (staff member) inputs by voice, "My blood sugar level was high yesterday."

[1837] Step 5:

[1838] The device converts the voice into text data, creating text data such as "yesterday's blood sugar level was high."

[1839] Step 6:

[1840] The terminal transmits the text data to the server.

[1841] Step 7:

[1842] The server updates the relevant patient information in the database.

[1843] Example 1

[1844] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1845] Conventional patient information management systems in nursing care facilities require staff to manually input and manage patient information, placing a heavy burden on staff and taking time to search for and update information.In addition, since information cannot be obtained or updated in real time via voice input, efficient information management is not possible in emergencies or when staff are busy.

[1846] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1847] In this invention, the server includes means for registering patient information in a database, means for acquiring patient identification information by voice recognition, means for acquiring corresponding patient information from the database based on the patient identification information, means for converting voice-input patient condition information into text and updating the corresponding patient information in the database, and means for processing the voice recognition results in real time and transmitting the acquired patient information to the wearable device, thereby enabling staff to efficiently manage patient information using voice input and acquire and update information in real time.

[1848] "Means for registering patient information in a database" refers to a system for storing information such as the patient's name, ID, medical history, recent condition, and precautions in a database.

[1849] The "means for acquiring patient identification information by voice recognition" is a mechanism for recognizing patient identification information input by voice, converting it into text data, and acquiring it.

[1850] The "means for obtaining relevant patient information from a database based on patient identification information" refers to a mechanism for searching a database based on identification information obtained by voice recognition and obtaining corresponding patient information.

[1851] "Means for outputting acquired patient information by voice or displaying it on a screen" refers to a system for conveying patient information acquired from a database to staff through voice or a display.

[1852] "Means for converting voice-input patient condition information into text and updating the corresponding patient information in the database" refers to a system in which staff converts new patient condition information entered by voice into text and updates the corresponding information in the database based on that information.

[1853] "Means for processing voice recognition results in real time and transmitting acquired patient information to a wearable device" refers to a mechanism for instantly processing the results obtained by voice recognition and transmitting acquired patient information based on the results to a wearable device.

[1854] The present invention relates to an information sharing system that enables staff in a nursing home to efficiently provide appropriate care to individual patients. The operation of the system of the present invention will be specifically described below.

[1855] System Overview

[1856] The system consists of the following elements:

[1857] 1. Database: Stores information such as patient name, ID, medical history, recent condition, and notes.

[1858] 2. Speech recognition device: Converts patient identification information entered by staff into text data using the Google Cloud Speech-to-Text API.

[1859] 3. Information acquisition device: Extracts relevant patient information from the database based on the converted text data.

[1860] 4. Audio output and display devices: Provide acquired patient information to staff. Use Web Speech API for audio output and a standard tablet or smartphone for display.

[1861] 5. Updater: Converts new patient status information entered by voice into text and updates the database.

[1862] Program processing

[1863] Patient information registration

[1864] 1. The user (administrator) uses a dedicated input form to enter information such as the patient's name, ID, medical history, recent condition, and precautions.

[1865] 2. The terminal packages the entered information and sends it to the server.

[1866] 3. The server stores the received information in the database, creating a new record for a new patient and updating the existing record for an existing patient.

[1867] Searching and retrieving patient information

[1868] 1. The user (staff member) speaks the patient's identification information into the wearable device.

[1869] 2. The device converts the voice input into text data in real time and sends the text data to the server.

[1870] 3. The server searches the database based on the received text data and retrieves the relevant patient information.

[1871] 4. The server sends the acquired patient information to the terminal in voice or text format.

[1872] 5. The terminal outputs the received patient information by voice or displays it on the display.

[1873] Patient Information Updates

[1874] 1. The user (staff member) speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID 1234").

[1875] 2. The device converts the voice into text data and sends it to the server.

[1876] 3. The server updates the relevant patient information in the database based on the text data received.

[1877] Specific examples

[1878] Example 1: Retrieving patient information

[1879] 1. The user (staff member) speaks into the wearable device, "Please tell me the information for ID1234."

[1880] 2. The device converts the voice to text and sends the data "Please tell me the information about ID1234" to the server.

[1881] 3. The server retrieves the patient information corresponding to "ID1234" from the database.

[1882] 4. The server converts the acquired information into audio data and sends it to the device.

[1883] 5. The device outputs a voice message saying, "Yamada Taro has diabetes. His blood sugar levels have been stable recently. He needs to be careful about his diet."

[1884] Example 2: Updating patient information

[1885] 1. The user (staff member) speaks into the wearable device, "Record the recent status of ID1234."

[1886] 2. The device converts the voice to text and sends the data "Record the recent status of ID1234" to the server.

[1887] 3. The device will say, "Status has been checked. Please tell us the latest status."

[1888] 4. The user (staff member) speaks, "My blood sugar level was high yesterday."

[1889] 5. The device converts the voice into text and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[1890] 6. The server updates the relevant patient information in the database.

[1891] As described above, the system of the present invention enables nursing home staff to obtain and update patient information in real time, enabling them to provide prompt and appropriate care.

[1892] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1893] Patient information registration

[1894] Step 1:

[1895] The user enters patient information using an input form. The information to be entered includes the patient's name, ID, medical history, recent condition, and precautions. Specifically, the user enters each item into the form on the web browser and clicks the submit button. The input data at this point is the string data for each item entered by the user.

[1896] Step 2:

[1897] The device packages the input information and sends it to the server. Specifically, JavaScript running on the device converts the input data into JSON format and sends it to the server via an HTTP POST request. The input is patient information in JSON format, and the output is an HTTP request.

[1898] Step 3:

[1899] The server stores the received information in the database. During this process, the server parses the HTTP POST request and extracts the JSON data. For new patients, it creates a new record using the SQL INSERT statement, and for existing patients, it updates the record using the UPDATE statement. The input is JSON-formatted data, and the output is the patient information stored in the database.

[1900] Searching and retrieving patient information

[1901] Step 1:

[1902] The user speaks the patient's identification information into the wearable device. Specifically, the user presses the voice input button and says, "Tell me the information about ID1234." The input data at this point is voice data.

[1903] Step 2:

[1904] The device converts voice input into text data in real time and sends that text data to the server. Specifically, a speech recognition engine (e.g., Google Cloud Speech-to-Text API) analyzes the voice data and generates text data such as "Please tell me the information about ID1234." This text data is then sent to the server as an HTTP request. The input is voice data, and the output is text data and an HTTP request.

[1905] Step 3:

[1906] The server searches the database based on the received text data and retrieves the corresponding patient information. Specifically, the server analyzes the text data and retrieves the patient information from the database using an SQL SELECT statement. The input is text data, and the output is the corresponding patient information.

[1907] Step 4:

[1908] The server acquires patient information and sends it to the terminal in voice or text format. The server converts the acquired information into JSON format and sends it to the terminal as an HTTP response. The input is patient information and the output is an HTTP response.

[1909] Step 5:

[1910] The terminal outputs the received patient information by voice or displays it on the display. Specifically, it uses a voice output library (e.g., SpeechSynthesis of the Web Speech API) to convey the information to the user by voice, or displays it as text on the display. The input is patient information in JSON format, and the output is voice data or display.

[1911] Patient Information Updates

[1912] Step 1:

[1913] The user speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID1234"). The input data is voice data.

[1914] Step 2:

[1915] The device converts the voice into text data and sends it to the server. Specifically, the voice recognition engine analyzes the voice data, generates text data such as "Record the recent status of ID1234," and sends it to the server as an HTTP request. The input is voice data, and the output is text data and an HTTP request.

[1916] Step 3:

[1917] The device will output a voice message saying, "The status has been checked. Please tell us the latest status." Specifically, a voice output library is used to play a voice message prompting the user for confirmation. The input is the text data mentioned earlier, and the output is a voice confirmation.

[1918] Step 4:

[1919] The user inputs by voice, "My blood sugar level was high yesterday." The input data is voice data.

[1920] Step 5:

[1921] The device again converts the voice into text data and sends it to the server. Specifically, the voice recognition engine generates text data such as "yesterday's blood sugar level was high" and sends it to the server as an HTTP request. The input is voice data, and the output is text data and an HTTP request.

[1922] Step 6:

[1923] The server updates the corresponding patient information in the database. Specifically, it uses the SQL UPDATE statement to update the corresponding database record based on the text data received by the server. The input is the text data, and the output is the updated database record.

[1924] (Application example 1)

[1925] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1926] In traditional nursing homes and factories, staff were required to efficiently obtain information about patients and equipment and respond appropriately. However, registering and updating information was time-consuming and there was a high possibility of input errors. In addition, it was difficult to obtain the necessary information in real time, which sometimes resulted in delayed or incorrect responses. This led to issues such as reduced operational efficiency at nursing homes and factories and inadequate management of patients and equipment.

[1927] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1928] In this invention, the server includes a means for registering patient information in a database, a means for acquiring patient identification information through voice recognition, a means for registering the operating status of multiple devices in the database, and a means for acquiring device identification information through voice recognition. This allows staff to quickly acquire patient and device information and respond appropriately. Furthermore, by enabling the registration and updating of information through voice recognition, it is possible to prevent input errors and improve operational efficiency.

[1929] "Patient information" refers to information such as the patient's name, ID, medical history, recent condition, and precautions.

[1930] A "database" is a collection of information designed to efficiently store specific information and to make it easy to search, update, and delete.

[1931] "Speech recognition" is a technology that converts speech into text data, analyzes speech input, and outputs a corresponding string of characters.

[1932] "Identification information" refers to information that uniquely identifies a specific individual or device, and includes an ID, name, etc.

[1933] "Audio output" is a technology that outputs audio data in a format that humans can hear through devices such as speakers.

[1934] "Display" is a technology for displaying information such as text and images on the screen of a display device.

[1935] "Device information" refers to information including the operating status of machines and robots, error information, maintenance information, and the like.

[1936] A "wearable device" is a computing device that can be worn by a user, including smart glasses and head-mounted displays.

[1937] The present invention relates to an information sharing system that enables staff in nursing homes and other care facilities, as well as in factories, to efficiently provide appropriate care and maintenance for individual patients and devices. The operation of the system of the present invention will be specifically described below.

[1938] System Overview

[1939] The system of the present invention comprises the following elements:

[1940] 1. Database

[1941] Stores patient and device information. Patient information includes name, ID, medical history, recent condition, and precautions, while device information includes operating status, error information, and maintenance information.

[1942] 2. Voice Recognition Device

[1943] The system converts patient and device identification information entered by staff via voice into text data using Python and SpeechRecognition (Google Speech-to-Text API).

[1944] 3. Information acquisition device

[1945] Based on the converted text data, the relevant patient and device information is extracted from the database using SQLite3.

[1946] 4. Audio output devices and display devices

[1947] The acquired patient and device information is provided to staff using speakers for audio output and tablets or smart glasses for display.

[1948] 5. Update device

[1949] Converts new spoken patient and device status information into text and updates the database. Uses Python, SpeechRecognition, and SQLite3.

[1950] Program processing

[1951] Patient information registration

[1952] The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions using a dedicated input form, and the terminal packages the entered information and sends it to the server. The server stores the received information in a database. At this time, a new record is created for new patients, and the corresponding record is updated for existing patients.

[1953] Registering device information

[1954] The user (administrator) enters information such as the device ID, operating status, error information, and maintenance information using a dedicated input form, and the terminal packages the entered information and sends it to the server. The server stores the received information in a database. At this time, a new record is created for new devices, and the corresponding record is updated for existing devices.

[1955] Search and retrieve patient and device information

[1956] The user (staff member) speaks into the wearable terminal to input the patient or device identification information, which the terminal converts into text data and sends to the server. The server searches the database based on the received text data and obtains the relevant patient and device information. The server then sends the obtained information to the terminal in voice or text format, and the terminal outputs the received information as voice or displays it on a display.

[1957] Specific examples

[1958] Example 1: Retrieving patient information

[1959] The user (staff member) speaks into the wearable device, saying, "Tell me the information about patient ID 1234." The device converts the speech into text and sends the data "Tell me the information about patient ID 1234" to the server. The server retrieves the patient information corresponding to "patient ID 1234" from the database. The server converts the retrieved information into voice data and sends it to the device. The device outputs a voice message saying, "Patient Name has diabetes. His blood sugar level has been stable recently. He needs to be careful about his diet."

[1960] Example 2: Updating Device Information

[1961] The user (staff member) speaks to the wearable device, saying, "The status of device ID 002 will be updated. The new status is an error, and the details are motor failure." The device converts the speech into text and sends the data "The status of device ID 002 will be updated. The new status is an error, and the details are motor failure" to the server. The server updates the relevant device information in the database, and the device outputs a voice message saying, "The status has been updated."

[1962] Prompt Sentence Examples

[1963] 1. "Please check the status of the following machine: Machine ID: 12345"

[1964] 2. "Please update the status of machine ID: 12345. New status: Under maintenance, Details: Part replaced"

[1965] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1966] Step 1:

[1967] The user inputs the patient or device identification information (e.g., patient ID or device ID) by voice. The input voice data is picked up by the microphone. The picked up voice data is sent by the terminal to the voice recognition engine (SpeechRecognition). The voice recognition engine converts the voice data into text data and outputs the text data to the terminal.

[1968] Input: Voice data (patient or device identification information)

[1969] Output: Text data (converted identification information)

[1970] Step 2:

[1971] The device receives the text data, packages it, and sends it to the server. The device converts the text data to JSON format and sends it as an HTTP request to the server. The server receives the HTTP request and prepares a database query.

[1972] Input: Text data of identification information

[1973] Output: HTTP request sent to the server

[1974] Step 3:

[1975] The server searches the database (SQLite3) based on the received text data and retrieves the corresponding patient or device information. The server executes an SQL query and retrieves the corresponding records. The retrieved information is converted to JSON format on the server and sent to the device as an HTTP response.

[1976] Input: The identification information sent to the server

[1977] Output: JSON data of the relevant patient or device information

[1978] Step 4:

[1979] The device analyzes the JSON data received from the server and provides information to the user through a voice output device or display device. The device parses the JSON data and converts it into a format for a speech synthesis engine or display. The speech synthesis engine converts the text data into audio data and plays it over the speaker. The display device displays the formatted text.

[1980] Input: JSON data from the server

[1981] Output: Audio output or display

[1982] Step 5:

[1983] When the user speaks a new patient or device status, the device again converts the voice to text and sends it to the server. The device then uses a speech recognition engine to convert the voice data to text, generates JSON data containing the updated information, and sends it to the server.

[1984] Input: Audio data (new state information)

[1985] Output: HTTP request sent to the server

[1986] Step 6:

[1987] The server updates the corresponding patient or device record in the database based on the received text data. The server executes the SQL UPDATE statement to update the corresponding record in the database with the new state. After the update is complete, the server returns a notification of update completion to the terminal.

[1988] Input: Text data of the update contents

[1989] Output: Notification of update completion

[1990] Step 7:

[1991] The device receives the update completion notification from the server and notifies the user by voice or display that the update is complete. A voice synthesis engine is used to play a message indicating that the update is complete, or a message is displayed on the screen.

[1992] Input: Notification of update completion from the server

[1993] Output: Update completion notification via audio output or display

[1994] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1995] This invention relates to an information sharing system that enables staff in nursing homes and other care facilities to efficiently provide appropriate care to each elderly person. By combining this system with an emotion engine that recognizes the user's emotions, it becomes possible to provide even more accurate care.

[1996] System Overview

[1997] The system of the present invention comprises the following elements:

[1998] 1. Database: Stores information such as patient name, ID, medical history, recent condition, and notes.

[1999] 2. Voice recognition device: Converts patient identification information entered by staff into text data.

[2000] 3. Information acquisition device: Extracts relevant patient information from the database based on the converted text data.

[2001] 4. Audio output and display devices: Provide retrieved patient information to staff.

[2002] 5. Updater: Converts new patient status information entered by voice into text and updates the database.

[2003] 6. Emotion Engine: Recognizes user emotions in real time and adjusts the information provided as needed.

[2004] Program processing

[2005] Patient information registration

[2006] 1. The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions into a dedicated form.

[2007] 2. The terminal packages the input information into a data package and sends it to the server.

[2008] 3. The server analyzes the received information and stores it in the database, creating a new record for a new patient and updating the existing record for an existing patient.

[2009] Searching and retrieving patient information

[2010] 1. The user (staff member) speaks the patient's identification information (e.g., ID1234) into the wearable device.

[2011] 2. The device converts the voice input into text data in real time and sends the text data to the server.

[2012] 3. The server searches the database based on the received text data and retrieves the relevant patient information.

[2013] 4. The server converts the acquired patient information into text or voice data and sends it to the terminal.

[2014] 5. The terminal outputs the received patient information by voice or displays it on the display.

[2015] Patient Information Updates

[2016] 1. The user (staff member) speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID 1234").

[2017] 2. The device converts the voice into text data and sends it to the server.

[2018] 3. The device will say, "Status has been checked. Please tell us the latest status."

[2019] 4. The user (staff member) speaks, "My blood sugar level was high yesterday."

[2020] 5. The device converts the voice into text data and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[2021] 6. The server updates the relevant patient information in the database.

[2022] Recognizing and modulating emotional information

[2023] 1. The device captures the user's (staff member's) voice and facial expressions in real time and sends them to the emotion engine.

[2024] 2. The emotion engine recognizes the user's (staff's) emotions from the received data and captures them as text or metadata.

[2025] 3. The emotion engine adjusts the content and expression of the patient information sent from the server based on the emotion.

[2026] 4. The server sends the adjusted patient information to the terminal and provides it to the staff in an appropriate format.

[2027] Specific examples

[2028] Example 1: Retrieving patient information

[2029] 1. The user (staff member) speaks into the wearable device, "Please tell me the information for ID1234."

[2030] 2. The device converts the voice to text and sends the data "Please tell me the information about ID1234" to the server.

[2031] 3. The server retrieves the patient information corresponding to "ID1234" from the database.

[2032] 4. The emotion engine analyzes the user's (staff's) current emotions and adjusts the way information is presented as needed (e.g., if the user is feeling stressed, it summarizes the output and presents it accordingly).

[2033] 5. The server converts the adjusted information into audio data and sends it to the device.

[2034] 6. The device outputs a voice message saying, "Yamada Taro has diabetes. His blood sugar levels have been stable recently. He needs to be careful about his diet."

[2035] Example 2: Updating patient information

[2036] 1. The user (staff member) speaks into the wearable device, "Record the recent status of ID1234."

[2037] 2. The device converts the voice to text and sends the data "Record the recent status of ID1234" to the server.

[2038] 3. The device will say, "Status has been checked. Please tell us the latest status."

[2039] 4. The user (staff member) speaks, "My blood sugar level was high yesterday."

[2040] 5. The device converts the voice into text and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[2041] 6. The sentiment engine analyzes the user's emotions and asks additional questions to ensure the input is accurate (e.g., "Was it really expensive?").

[2042] 7. The server updates the patient information in the database.

[2043] As described above, the system of the present invention enables nursing home staff to provide prompt and appropriate care to each elderly person, and by incorporating the user's emotions, the accuracy of information and the quality of care can be improved.

[2044] The processing flow will be explained below.

[2045] Patient information database registration

[2046] Step 1:

[2047] The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions into a dedicated form.

[2048] Step 2:

[2049] The terminal converts the input information into a data package and transmits it to the server.

[2050] Step 3:

[2051] The server analyzes the received information and stores it in the database. For new patients, it creates a new record, and for existing patients, it updates the corresponding record.

[2052] Searching and retrieving patient information

[2053] Step 1:

[2054] The user (staff member) speaks the patient's identification information (e.g., ID1234) into the wearable device.

[2055] Step 2:

[2056] The device converts the voice input into text data in real time and sends the text data to the server.

[2057] Step 3:

[2058] The server searches the database based on the received text data and retrieves the relevant patient information.

[2059] Step 4:

[2060] The device captures the user's (staff member's) voice and facial expressions and sends them to the emotion engine.

[2061] Step 5:

[2062] The emotion engine recognizes the user's (staff's) emotions from the received data and generates emotion information.

[2063] Step 6:

[2064] The server adjusts the acquired patient information based on the emotion information and summarizes the information as necessary.

[2065] Step 7:

[2066] The server converts the adjusted patient information into voice or text data and transmits it to the terminal.

[2067] Step 8:

[2068] The patient information received by the terminal is output as voice or displayed on the screen.

[2069] Patient Information Updates

[2070] Step 1:

[2071] The user (staff member) speaks into the wearable device, "Record the recent status of ID1234."

[2072] Step 2:

[2073] The device converts the voice into text data and sends it to the server.

[2074] Step 3:

[2075] The device will output a voice message saying, "Status has been checked. Please tell us the latest status."

[2076] Step 4:

[2077] The user (staff member) inputs by voice, "My blood sugar level was high yesterday."

[2078] Step 5:

[2079] The device converts the voice into text data, generating text data such as "yesterday's blood sugar level was high."

[2080] Step 6:

[2081] The terminal transmits the text data to the server.

[2082] Step 7:

[2083] The server analyzes the text data and updates the relevant patient information in the database.

[2084] Step 8:

[2085] The device captures the user's (staff member's) voice and facial expressions and sends them to the emotion engine.

[2086] Step 9:

[2087] The emotion engine recognizes the user's (staff's) emotions from the received data and adds questions to confirm the accuracy of the input.

[2088] Step 10:

[2089] The terminal transmits the re-entered information to the server.

[2090] Step 11:

[2091] The server will then finalize the patient information in the database and update it if necessary.

[2092] Example 2

[2093] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2094] Conventional information sharing systems in nursing homes mainly involve managing patient information manually, which often lacks efficiency and accuracy. Furthermore, information is provided without taking into account the emotions and stress levels of users (staff), making it impossible to improve the quality of care. It is necessary to provide a system that solves these problems and provides prompt and appropriate care to each elderly person.

[2095] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2096] In this invention, the server includes means for registering patient information in a database, means for acquiring patient identification information by voice recognition, means for acquiring corresponding patient information from the database based on the patient identification information, means for outputting the acquired patient information by voice or displaying it on a screen, means for adjusting the content and presentation method of information using an emotion engine that recognizes user emotions, means for converting voice-input patient condition information into text and updating the corresponding patient information in the database, means for processing the voice recognition results in real time and transmitting the acquired patient information to a wearable device, means for capturing the user's voice and facial expressions and transmitting them to the emotion engine, and means for adjusting the voice or text output based on the user's emotions. This allows information to be registered, acquired, and updated in the database accurately and efficiently, and by taking the user's emotions into consideration, it is possible to improve the quality of care.

[2097] The "means for registering patient information in a database" is a component that performs processing to store information such as the patient's name, ID, medical history, recent condition, and points to note in a database system.

[2098] The "means for acquiring patient identification information by voice recognition" is a mechanism for acquiring patient identification information input by voice as text data using voice recognition technology.

[2099] The "means for acquiring corresponding patient information from a database based on patient identification information" is a component that performs processing to search for and extract corresponding patient information from within a database using the acquired identification information.

[2100] "Means for outputting acquired patient information by voice or displaying it on a screen" refers to a mechanism for outputting or displaying information by voice or on a screen in order to convey searched or acquired patient information to the user.

[2101] The "emotion engine that recognizes user emotions" is a technology and software that analyzes and recognizes emotions from the user's voice and facial expressions in real time.

[2102] The "means for converting voice-inputted patient condition information into text and updating the corresponding patient information in the database" is a component that converts voice-inputted information into text and performs the process of updating the patient information stored in the database to the latest state.

[2103] The "means for processing voice recognition results in real time and transmitting the acquired patient information to a wearable terminal" is a mechanism for processing text data obtained by voice recognition in real time and transmitting the corresponding patient information to a wearable terminal.

[2104] The "means for capturing the user's voice and facial expression and transmitting it to the emotion engine" is a system for capturing the user's voice and facial expression and transmitting that data to the emotion engine.

[2105] "Means for adjusting voice or text output based on user emotions" refers to technology for adjusting the content and expression of the voice or text to be output in accordance with the user's emotions recognized by the emotion engine.

[2106] This invention relates to an information sharing system that enables staff in nursing homes and other care facilities to efficiently provide appropriate care to each elderly person. This system registers patient information in a database, acquires patient identification information using voice recognition technology, retrieves relevant patient information from the database based on that information, and outputs the acquired patient information as voice output or on a display. Furthermore, it differs from conventional systems in that it can recognize the user's emotions and adjust the content and presentation method of information based on those emotions.

[2107] The main components of the system are as follows:

[2108] Hardware and Software Components

[2109] 1. Database:

[2110] Store information such as the patient's name, ID, medical history, recent condition, and precautions. For example, use a relational database such as MySQL.

[2111] 2. Voice recognition device:

[2112] The patient's identification information is converted into text data by staff using the Google Cloud Speech-to-Text API.

[2113] 3. Information acquisition device:

[2114] Based on the converted text data, the relevant patient information is extracted from the database using a script (e.g., Python) that runs on the server.

[2115] 4. Audio output and display devices:

[2116] Use voice output devices and display terminals to provide retrieved patient information to staff. Examples include smartwatches and tablets.

[2117] 5. Update device:

[2118] New patient status information is spoken into the database and converted to text, using the Google Cloud Speech-to-Text API as well as a speech recognizer.

[2119] 6. Emotion Engine:

[2120] It recognizes the user's emotions in real time and adjusts the information provided as needed. As a concrete example, it uses the Microsoft Azure Emotion API.

[2121] Examples of concrete examples and prompts

[2122] 1. Obtaining patient information

[2123] The user (staff member) speaks into the wearable device, "Please tell me the information for ID1234."

[2124] The device converts the voice into text and sends the data "Please tell me the information about ID1234" to the server.

[2125] The server retrieves the patient information corresponding to "ID1234" from the database.

[2126] The emotion engine analyzes the user's (staff's) current emotions and adjusts the way information is presented as needed (e.g., if the user is feeling stressed, it summarizes the output and presents it accordingly).

[2127] The server converts the adjusted information into audio data and transmits it to the terminal.

[2128] The device outputs a voice message saying, "Yamada Taro has diabetes. His blood sugar levels have been stable recently. He needs to be careful about his diet."

[2129] 2. Updating patient information

[2130] The user (staff member) speaks into the wearable device, "Record the recent status of ID1234."

[2131] The device converts the voice into text and sends the data "Record the recent status of ID1234" to the server.

[2132] The device will output a voice message saying, "Status has been checked. Please tell us the latest status."

[2133] The user (staff member) inputs by voice, "My blood sugar level was high yesterday."

[2134] The device converts the voice into text and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[2135] The server updates the relevant patient information in the database.

[2136] By using these specific procedures and techniques, the present invention is designed to enable nursing home staff to provide more efficient and accurate care. As described above, the present invention enables the registration, acquisition, and updating of patient information quickly and accurately, and further improves the quality of care by taking into account the user's emotions.

[2137] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2138] Patient information registration

[2139] Step 1:

[2140] The user (administrator) enters information such as the patient's name, ID, medical history, recent condition, and precautions into a dedicated form on the browser.

[2141] Specifically, the administrator enters information such as "Yamada Taro," "ID 1234," "has had diabetes for 10 years," "his blood sugar levels are stable these days," and "pay attention to dietary management."

[2142] Step 2:

[2143] The terminal converts the input information into a JSON format data package and sends it to the server.

[2144] Specifically, the device processes the input information into data such as "{"name":"Yamada Taro","ID":"ID1234","Medical history":"Diabetes for 10 years","Recent condition":"Stable blood sugar","Points to note":"Dietary management"}" and sends it to the server.

[2145] Step 3:

[2146] The server analyzes the received JSON data and stores it in the database.

[2147] Specifically, the server inserts a new record into the "patients" table or "updates" an existing record. Based on the JSON data as input, it generates a status indicating the data has been saved to the database as output.

[2148] Searching and retrieving patient information

[2149] Step 1:

[2150] The user (staff member) speaks into the wearable device to input the patient's identification information (e.g., "Please tell me the information for ID1234").

[2151] Specifically, the staff member speaks into the smartwatch, saying, "Tell me the information for ID1234."

[2152] Step 2:

[2153] The device converts the voice input into text data in real time and sends the text data to the server.

[2154] Specifically, the device converts the voice into text data such as "Tell me the information about ID1234" and sends it to the server.

[2155] Step 3:

[2156] The server queries the database based on the received text data and retrieves the corresponding patient information.

[2157] Specifically, the server executes the query "SELECT FROM patients WHERE ID='ID1234'." Based on the text data as input, corresponding patient information is obtained as output.

[2158] Step 4:

[2159] The server processes the acquired patient information into JSON format data and sends it to the terminal.

[2160] Specifically, the server generates the data "{"name":"Yamada Taro","ID":"ID1234","Medical history":"Diabetes for 10 years","Recent condition":"Stable blood sugar","Cautions":"Dietary management"}" and sends it to the terminal.

[2161] Step 5:

[2162] The patient information received by the terminal is output as voice or displayed on the screen.

[2163] Specifically, the device will output a message in voice or display on the screen saying, "Yamada Taro has diabetes. His blood sugar levels have been stable recently. He needs to be careful about his diet."

[2164] Patient Information Updates

[2165] Step 1:

[2166] The user (staff member) speaks into the wearable device to input a new patient status (e.g., "Record the latest status of ID 1234").

[2167] As for specific actions, the staff member said, "We will record the recent status of ID1234."

[2168] Step 2:

[2169] The device converts the voice into text data and sends it to the server.

[2170] Specifically, the terminal converts this into text data that reads "Record the recent status of ID1234" and sends it to the server.

[2171] Step 3:

[2172] The device will output a voice message saying, "Status has been checked. Please tell us the latest status."

[2173] Specifically, the device will announce in a voice message, "Status has been checked. Please tell us the latest status."

[2174] Step 4:

[2175] The user (staff member) inputs by voice, "My blood sugar level was high yesterday."

[2176] Specifically, the staff member says, "yesterday's blood sugar level was high."

[2177] Step 5:

[2178] The device converts the voice into text data and sends the text data, such as "yesterday's blood sugar level was high," to the server.

[2179] Specifically, the device converts the text "yesterday's blood sugar level was high" and sends it to the server.

[2180] Step 6:

[2181] The server updates the relevant patient information in the database.

[2182] Specifically, the server executes the query "UPDATE patients SET recent status = 'high blood glucose level' WHERE ID = 'ID1234'". Based on the text data as input, the database is updated as output.

[2183] Recognizing and modulating emotional information

[2184] Step 1:

[2185] The device captures the user's (staff member's) voice and facial expressions in real time and sends them to the emotion engine.

[2186] Specifically, the device uses its built-in camera and microphone to capture the facial expressions and voice tone of the staff member.

[2187] Step 2:

[2188] The emotion engine recognizes the user's (staff's) emotions from the received data and captures them as text or metadata.

[2189] Specifically, the emotion engine generates emotion data such as "feeling stressed."

[2190] Step 3:

[2191] The emotion engine adjusts the content and expression of the patient information sent from the server based on the emotion.

[2192] Specifically, the emotion engine instructs the server to "summarize and present information to reduce stress."

[2193] Step 4:

[2194] The server sends the adjusted patient information to the terminal and provides it to the staff in an appropriate format.

[2195] Specifically, the server sends adjusted information to the terminal, such as "Yamada Taro has diabetes. He needs to be careful."

[2196] The above are the specific processing steps of the system, and the detailed operations and data processing / calculation contents at each step.

[2197] (Application example 2)

[2198] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2199] Nursing care facilities require an information sharing system that enables staff to provide prompt and appropriate care to patients. However, with existing systems, staff fatigue and stress can affect the quality of care, and the method of presenting information is not sufficiently optimized. Furthermore, there is a high risk of input errors when updating information via voice input, making information reliability an issue. This hinders the consistency and efficiency of the quality of care.

[2200] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2201] In this invention, the server includes means for registering patient information in a database, means for acquiring patient identification information by voice recognition, means for acquiring corresponding patient information from the database based on the patient identification information, means for recognizing staff emotions using an emotion engine, means for adjusting the method of providing patient information based on the recognized emotion, means for converting voice-input patient condition information into text and updating the corresponding patient information in the database, and means for processing the voice recognition results in real time and transmitting the acquired patient information to the wearable device. This makes it possible to optimize the method of providing information taking into account the emotions of staff and efficiently provide appropriate care to patients.

[2202] "Patient information" refers to data about individual patients, such as their name, ID, medical history, recent condition, and precautions.

[2203] A "database" is a collection of information that stores patient information in an organized manner and that can be quickly retrieved and updated as needed.

[2204] "Speech recognition" is a technology that converts speech into text data in real time.

[2205] "Patient identification information" refers to information for individually identifying a patient, and includes ID, name, etc.

[2206] "Audio output" is a means for providing acquired information to the user in audio form.

[2207] "Display" refers to a means for visually presenting acquired information to the user.

[2208] An "emotion engine" is a technology that recognizes a user's emotions by analyzing data such as voice and facial expressions.

[2209] A "wearable terminal" is an electronic device that can be worn by a user and is capable of receiving and outputting information.

[2210] "Real time" means that processing occurs the instant an event occurs.

[2211] MODE FOR CARRYING OUT THE INVENTION

[2212] System Configuration

[2213] The present invention is an information sharing system that comprises the following elements:

[2214] 1. Database:

[2215] Stores information such as patient name, ID, medical history, recent status, and precautions.

[2216] 2. Voice recognition device:

[2217] This device converts patient identification information entered by staff into text data. Specifically, smart glasses or wearable devices capture voice and convert it into text in real time.

[2218] 3. Information acquisition device:

[2219] Extract relevant patient information from the database. Search the database based on the text data converted by voice recognition to obtain the necessary patient information.

[2220] 4. Audio output and display devices:

[2221] This device outputs acquired patient information as voice or displays it on a display. It converts text data of patient information into voice data and provides it to the user via a wearable device.

[2222] 5. Update device:

[2223] This system converts new patient status information entered by voice into text and updates the database. Specifically, it has the function of sending the information entered by staff via voice to the server as text data and updating the corresponding record in the database.

[2224] 6. Emotion Engine:

[2225] This engine captures the user's emotions in real time from their voice and facial expressions, and adjusts the way information is presented as needed. This allows it to maintain the quality of care by adjusting the way information is presented if staff are under high stress or fatigue.

[2226] Example of a system

[2227] Example 1: Retrieving patient information

[2228] 1. The user (staff member) speaks into the smart glasses, "Please tell me the patient's ID number."

[2229] 2. The device converts the speech into text and sends it to the server.

[2230] 3. The server retrieves the relevant patient information from the database and analyzes the staff's emotions through the emotion engine.

[2231] 4. The emotion engine assesses whether stress is present and summarizes the information provided if necessary.

[2232] 5. The device outputs the information it has acquired in voice (e.g., "You are a diabetic patient. Your condition has recently stabilized.").

[2233] Example 2: Updating patient information

[2234] 1. The user (staff member) speaks into the smart glasses, "I would like to record the patient's condition."

[2235] 2. The device converts the speech into text and sends it to the server.

[2236] 3. The server retrieves the patient's information from the database and outputs a confirmation message.

[2237] 4. The user (staff member) then voice-inputs, "My blood pressure was high."

[2238] 5. The emotion engine analyzes the user's emotions and confirms the input.

[2239] 6. Update the relevant patient information in the database and notify the patient by voice message that "Information has been updated."

[2240] Hardware and software used

[2241] Hardware:

[2242] Smart glasses: capable of voice input and output

[2243] Microphone: Data acquisition for voice recognition

[2244] Camera: Data acquisition for emotion recognition

[2245] software:

[2246] Python: used as a programming language

[2247] SQLite3: Database Management

[2248] speech_recognition: Speech recognition engine

[2249] pyttsx3: Audio output engine

[2250] emotion_recognition (tentative): Emotion recognition engine

[2251] Prompt Sentence Examples

[2252] Emotion Engine Input:

[2253] python

[2254] emotion_engine.recognize("audio_data_file_path");

[2255] emotion_engine.recognize("video_data_file_path");

[2256] Speech recognition engine prompts:

[2257] python

[2258] recognizer.recognize_google(audio, language='ja-JP');

[2259] Using these technologies, the system provides powerful support for staff to provide patient care quickly and efficiently.

[2260] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2261] Program processing steps

[2262] Example 1: Retrieving patient information

[2263] Step 1:

[2264] The user (staff member) speaks into the smart glasses and says, "Please tell me the patient's ID number."

[2265] Input: Audio data

[2266] How it works: The microphone in the smart glasses captures audio.

[2267] Output: Captured audio data

[2268] Step 2:

[2269] The device calls the speech recognition engine (speech_recognition) to convert the voice data into text data.

[2270] Input: Audio data

[2271] How it works: A speech recognition engine converts voice data into text.

[2272] Output: Text data (e.g., "What is the patient's ID number?")

[2273] Step 3:

[2274] The terminal transmits the text data to the server, and the server retrieves the corresponding patient information from the database.

[2275] Input: Text data

[2276] Operation: Based on the text data, a database is queried to retrieve patient information.

[2277] Output: Patient information data

[2278] Step 4:

[2279] The server uses an emotion engine to recognize the emotions of staff and adjust the way information is presented.

[2280] Input: Patient information data, voice and facial expression data

[2281] How it works: The emotion engine analyzes voice and facial expression data to recognize the user's emotions, adjusting the patient summary and detailed information as needed.

[2282] Output: Reconciled patient information data

[2283] Step 5:

[2284] The terminal calls the voice output engine (pyttsx3) to output the adjusted patient information as voice.

[2285] Input: Adjusted patient information data

[2286] How it works: The voice output engine converts text data into voice data and outputs it from the speaker.

[2287] Output: Speech data (e.g., "I am a diabetic patient. My condition has stabilized recently.")

[2288] Example 2: Updating patient information

[2289] Step 1:

[2290] The user (staff member) speaks into the smart glasses and says, "I would like to record the patient's condition."

[2291] Input: Audio data

[2292] How it works: The microphone in the smart glasses captures audio.

[2293] Output: Captured audio data

[2294] Step 2:

[2295] The device calls the speech recognition engine (speech_recognition) to convert the voice data into text data.

[2296] Input: Audio data

[2297] How it works: A speech recognition engine converts voice data into text.

[2298] Output: Text data (e.g., "I want to record the patient's condition")

[2299] Step 3:

[2300] The terminal sends the text data to the server, which then retrieves the patient's information from the database and outputs a confirmation message.

[2301] Input: Text data

[2302] Behavior: Query database and generate confirmation message based on text data

[2303] Output: Text data of confirmation message

[2304] Step 4:

[2305] The device calls the voice output engine (pyttsx3) to output the confirmation message as voice.

[2306] Input: Text data of confirmation message

[2307] How it works: The voice output engine converts text data into voice data and outputs it from the speaker.

[2308] Output: Audio data (e.g., "We would like to record the patient's status. Do you wish to continue?")

[2309] Step 5:

[2310] The user (staff member) then inputs "My blood pressure was high" by voice.

[2311] Input: Audio data

[2312] How it works: The microphone in the smart glasses captures audio.

[2313] Output: Captured audio data

[2314] Step 6:

[2315] The terminal converts the voice data into text data and sends it to the server.

[2316] Input: Audio data

[2317] How it works: A speech recognition engine converts voice data into text.

[2318] Output: Text data (e.g., "My blood pressure was high")

[2319] Step 7:

[2320] The server uses an emotion engine to analyze the user's emotions and confirm the input content.

[2321] Input: Text data, voice and facial expression data

[2322] How it works: The emotion engine analyzes voice and facial expression data to recognize the user's emotions and generates a confirmation message if necessary.

[2323] Output: Text data of confirmation message (e.g., "Was your blood pressure really high?")

[2324] Step 8:

[2325] The relevant patient information in the database is updated and a voice message is generated saying, "The information has been updated."

[2326] Input: Verified text data

[2327] Behavior: Updates the corresponding record in the database and generates a confirmation message

[2328] Output: Text data of confirmation message

[2329] Step 9:

[2330] The device calls the voice output engine (pyttsx3) to output the confirmation message as voice.

[2331] Input: Text data of confirmation message

[2332] How it works: The voice output engine converts text data into voice data and outputs it from the speaker.

[2333] Output: Audio data (e.g. "Information has been updated")

[2334] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2335] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2336] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2337] The emotion identification model 59 as an emotion engine may determine the user's emotion accor...

Claims

1. a means for registering patient information in a database; means for obtaining patient identification information by voice recognition; means for retrieving corresponding patient information from a database based on the patient's identification information; a means for outputting the acquired patient information by voice or displaying it on a display; A system including:

2. 2. The system according to claim 1, further comprising means for converting voice-inputted patient condition information into text and updating the corresponding patient information in the database.

3. The system of claim 1 , further comprising: means for processing the speech recognition results in real time and transmitting the acquired patient information to the wearable terminal.

Images