System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving a language preference of a patient; means for translating medical information based on a generative model; means for displaying translation results in real time; means for a medical professional to review and modify the translation results; means for protecting the medical information; and means for applying an algorithm that takes cultural considerations into account.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The purpose of this invention is to eliminate communication barriers between patients and medical professionals. In particular, there are challenges in accurately and quickly conveying information to patients who speak different languages ​​and to elderly patients in medical settings. It is also necessary to protect patients' personal information and medical data, and to consider cultural differences. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system including the following means:

[0006] 1. A means of receiving the patient's language preference;

[0007] 2. A means of translating medical information based on generative models;

[0008] 3. A means to display translation results in real time;

[0009] 4. A means for medical professionals to review and correct the translation;

[0010] 5. Measures to protect health information;

[0011] 6. Means of applying culturally sensitive algorithms;

[0012] This system will enable patients and elderly people who speak different languages ​​to exchange information smoothly at medical facilities, while also ensuring the accuracy and reliability of medical information.

[0013] "Means for receiving patient language settings" refers to the function of the system providing an interface for the user (patient) to input or select language setting information to be used and acquiring the relevant information.

[0014] "Means for translating medical information based on generative models" refers to the ability to use generative models of artificial intelligence (AI) or machine learning to accurately translate input medical information from one language to another in real time.

[0015] "Means for displaying translation results in real time" refers to a function that immediately transmits the results of translation by the generative model to a display device or interface, allowing users (patients) and medical professionals to view the translation information in real time.

[0016] "Means for medical professionals to check and correct translation results" refers to a function that provides an interface for medical professionals to check the translation results provided by the generative model and make corrections or completions as necessary.

[0017] "Measures to protect medical information" refers to functions to maintain the confidentiality and security of data by encrypting patient personal information and medical data and adopting secure communication protocols.

[0018] "Means for applying culturally sensitive algorithms" refers to the ability to apply algorithms to generative models using training data provided by cultural experts to produce translations that are adapted to different cultural backgrounds and sensitivities. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The present invention is a system for effective communication between patients and medical professionals, and is designed to enable patients who speak different languages ​​and the elderly to smoothly communicate and understand information in medical settings.

[0041] The program of this system mainly consists of the following steps:

[0042] User Registration and Authentication

[0043] 1. User: Accesses the system and enters the required information (name, contact information, password, etc.) on the account registration screen.

[0044] 2. Terminal: Sends the entered user information to the server.

[0045] 3. Server: Stores user information in a database and generates and manages authentication information, which is used for subsequent logins.

[0046] Verifying your language preferences and needs

[0047] 1. User: After logging in, select the language you want to use on the language settings screen. Also, specify whether you need medical translation.

[0048] 2. Terminal: Sends the selected language setting information to the server.

[0049] 3. Server: Based on the received language settings, loads the appropriate medical dictionary and medical database.

[0050] Loading the medical terminology database

[0051] 1. Server: Loads appropriate medical dictionaries and medical databases and integrates them into the generative model, enabling professional medical information translation.

[0052] Real-time conversation translation

[0053] 1. User (Patient): During a consultation, the patient enters symptoms or questions via voice or text (e.g., "I've had a bad headache lately").

[0054] 2. Terminal: Sends input voice and text data to the server.

[0055] 3. Server: The generative model processes the received data and translates it in real time into the selected language (e.g., "I have been experiencing severe headaches recently").

[0056] 4. Terminal: Displays the translation results on the screen of the user (medical professional) and patient.

[0057] Check and complete translation results

[0058] 1. User (medical professional): Check the displayed translation results and make corrections or additions as necessary. This allows for correction of mistranslations and missing information.

[0059] 2. Terminal: Send the corrected information back to the server.

[0060] 3. Server: The generative model retranslates the corrected information and provides it to patients and specialists.

[0061] Information Protection and Security

[0062] 1. Server: All patient information and medical data is encrypted and protected using secure communication protocols.

[0063] 2. Device: Configured to ensure secure data communications.

[0064] Cultural sensitivity

[0065] 1. Server: Implements the algorithm that applies training data from cultural experts to the generative model and produces culturally sensitive translations.

[0066] 2. Generative model: Reflects cultural considerations when they are necessary for the translation results.

[0067] Specific examples

[0068] Scenario: An English-speaking foreign patient visits a Japanese hospital

[0069] 1. The user (patient) accesses the system and registers as a user.

[0070] 2. The device sends the registration information to the server, which stores the information in a database.

[0071] 3. The user (patient) selects "English" in the language settings, and the device sends that information to the server.

[0072] 4. The server loads the English-Japanese medical terminology dictionary and integrates it into the generative model.

[0073] 5. During a consultation, the user (patient) speaks, "I have been experiencing severe headaches for the past week."

[0074] 6. The device sends the voice data to the server, and the generative model translates it into Japanese as "I've been experiencing a severe headache since last week."

[0075] 7. The terminal displays the translation results on the user's (medical professional's) screen.

[0076] 8. The user (medical professional) reviews the translation and asks any additional questions.

[0077] 9. The server encrypts all data and uses secure communication protocols.

[0078] 10. The generative model appropriately reflects cultural sensitivity where necessary.

[0079] This system will enable elderly and foreign patients to receive medical care smoothly and ensure the accuracy and reliability of medical information.

[0080] The processing flow will be explained below.

[0081] Step 1:

[0082] User: Accesses the system and enters name, contact information, password, etc. on the account registration screen.

[0083] Step 2:

[0084] Terminal: Sends the entered registration information to the server.

[0085] Step 3:

[0086] Server: Stores the submitted information in a database and generates authentication information that the user will use the next time they log in.

[0087] Step 4:

[0088] User: After logging in, select the language to use on the language setting screen. For example, select "English."

[0089] Step 5:

[0090] Terminal: Sends the selected language setting information to the server.

[0091] Step 6:

[0092] Server: Loads the appropriate medical dictionary and medical database based on the selected language settings.

[0093] Step 7:

[0094] Server: Prepares for translation by integrating the loaded medical dictionary and medical database into the generative model.

[0095] Step 8:

[0096] User (Patient): During the consultation, the patient will voice or text in their symptoms and questions (e.g., "I have been experiencing severe headaches for the past week").

[0097] Step 9:

[0098] Terminal: Sends input voice and text data to the server.

[0099] Step 10:

[0100] Server: The generative model translates the received data into the specified language (e.g., Japanese) (e.g., "I have been experiencing severe headaches since last week").

[0101] Step 11:

[0102] Terminal: Displays the translation results in real time on the user's (medical professional's) screen.

[0103] Step 12:

[0104] User (medical professional): Checks the displayed translation and makes corrections or completions as necessary, for example, checking for additional symptoms.

[0105] Step 13:

[0106] Terminal: Send the corrected information back to the server.

[0107] Step 14:

[0108] Server: The generative model re-translates the corrected data and generates the final translation result.

[0109] Step 15:

[0110] Terminal: displays the final translation results on the screen of the user (patient) and medical professional.

[0111] Step 16:

[0112] Server: Protects patient personal information and medical data by encrypting it and applying secure communication protocols.

[0113] Step 17:

[0114] Server: Implements the algorithm that applies training data provided by cultural experts to the generative model and performs translation with cultural sensitivity.

[0115] Step 18:

[0116] Generative model: Properly reflecting the content of translations that require cultural sensitivity.

[0117] Through the above steps, the system of the present invention creates an environment in which elderly and foreign patients can receive medical care smoothly, and ensures the accuracy and reliability of medical information.

[0118] Example 1

[0119] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0120] Patients and the elderly who speak different languages ​​often have difficulty communicating and understanding information smoothly in medical settings. In particular, inaccurate medical terminology and explanations of symptoms can hinder diagnosis and treatment. Furthermore, translation errors and a lack of cultural sensitivity can lead to misunderstandings between patients and medical professionals.

[0121] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0122] In this invention, the server includes means for receiving a patient's language setting, means for translating medical information based on a generative AI model, means for displaying the translation results in real time, means for medical professionals to review and correct the translation results, means for protecting the medical information, means for applying a culturally sensitive algorithm, means for converting voice data to text, means for loading a medical terminology dictionary and a medical database, and means for smooth information exchange between patients and medical professionals. This enables patients and elderly people who speak different languages ​​to exchange information accurately and quickly in medical settings, improving the accuracy and reliability of medical information transmission.

[0123] The "means for receiving patient language setting" is an interface through which the system inputs and acquires information specifying the language used by the patient.

[0124] A "means for translating medical information based on a generative AI model" is a function that uses an artificial intelligence model to translate input medical-related information into different languages.

[0125] "Means for displaying the translation results in real time" refers to a technique for instantly displaying the translated information on the user's display device.

[0126] "Means for medical professionals to review and correct translation results" refers to an interface that allows medical professionals to review the translation results provided by the system and make corrections or additions as necessary.

[0127] "Measures to protect medical information" are encryption technologies and security protocols used to protect medical information, such as patient personal information and medical records, from third parties.

[0128] "Means for applying culturally sensitive algorithms" are algorithms that allow the generative AI model to take cultural background and specific context into account when translating.

[0129] "Means for converting voice data to text" refers to speech recognition technology that analyzes voice input and provides the content to the system as text data.

[0130] The "means for loading medical terminology dictionaries and medical databases" refers to a function for loading dictionaries and databases containing medical terminology and related information into the system.

[0131] "Means for smooth information exchange between patients and medical professionals" refers to interfaces and communication technologies that allow patients and medical professionals to organize information and communicate it quickly and accurately.

[0132] The present invention relates to a system for effective communication between patients and medical professionals. It is designed to enable patients who speak different languages ​​and elderly people to smoothly communicate and understand information in medical settings. The system is realized using the following main hardware and software:

[0133] Hardware and software used

[0134] Server: A high-performance backend server for handling databases, generative AI models, and encryption technologies.

[0135] Device: Computers and smart devices (e.g., PCs, tablets, smartphones) used by patients and healthcare professionals.

[0136] Generative AI models: Use OpenAI GPT-4 or similar language models to translate medical information.

[0137] Speech recognition engine: A technology that converts speech to text, such as the Google Cloud Speech-to-Text API.

[0138] Database: A relational database such as MySQL or PostgreSQL to manage user information and a medical dictionary.

[0139] Front-end technology: Frameworks for creating user interfaces, such as React and Vue.js.

[0140] Program processing flow

[0141] The system consists of the following steps:

[0142] 1. User Registration and Authentication

[0143] 1. The user (patient) accesses the system and enters the required information such as name, contact information, password, etc. The entered information is sent to the server using a secure communication protocol (SSL / TLS).

[0144] 2. The server stores the received user information in a database and generates a unique identifier (user ID).

[0145] 3. The server generates authentication information and manages the user account. This authentication information is used for subsequent logins.

[0146] 2. Identifying language preferences and needs

[0147] 1. After logging in, the user selects the language they want to use on the language settings screen. For example, they can choose English as their primary language. They can also specify whether they need medical translation.

[0148] 2. The terminal sends the selected language setting information to the server, and the server loads the appropriate medical terminology dictionary and medical database based on that information.

[0149] 3. Loading the Medical Terminology Database

[0150] 1. The server loads an open-source medical dictionary (e.g., MedDRA or SNOMED CT) according to the user's language settings.

[0151] 2. The server integrates the loaded dictionaries and databases into a generative AI model, enabling the translation of specialized medical information.

[0152] 4. Real-time translation of conversations

[0153] 1. The user (patient) enters symptoms and questions by voice or text during a consultation. For example, the user might enter "I have been experiencing severe headaches for the past week."

[0154] 2. The device sends the voice data to the server, which uses a speech recognition engine to convert the voice into text.

[0155] 3. The server uses the generative AI model to translate the text data into the selected language (e.g., Japanese) in real time.

[0156] 4. The terminal displays the translation result on the screen of the user (medical professional) and the patient. For example, the translation may read, "I have been experiencing a severe headache since last week."

[0157] 5. Check and complete the translation result

[0158] 1. The user (medical professional) checks the displayed translation and makes corrections or additions as necessary. For example, adding an additional question such as "When did it start?"

[0159] 2. The terminal sends the corrected information back to the server, and the server performs a retranslation.

[0160] 6. Data Protection and Security

[0161] 1. The server encrypts all patient information and medical data using standard encryption techniques such as AES.

[0162] 2. The server protects data using a secure communication protocol (SSL / TLS) and achieves encrypted communication using public and private keys.

[0163] 3. The device is configured to ensure secure data communications.

[0164] 7. Cultural Sensitivity

[0165] 1. The server applies training data from cultural experts to a generative AI model, implementing an algorithm that produces translations that reflect cultural nuances.

[0166] 2. The generative model will incorporate cultural considerations where necessary, for example, translating with consideration for expressions and etiquette specific to Japanese culture.

[0167] Specific examples

[0168] Scenario: An English-speaking foreign patient visits a Japanese hospital

[0169] 1. The user (patient) accesses the system and registers as a user.

[0170] 2. The device sends the registration information to the server, which stores the information in a database.

[0171] 3. The user (patient) selects "English" in the language settings, and the device sends that information to the server.

[0172] 4. The server loads an English-Japanese medical terminology dictionary and integrates it into the generative AI model.

[0173] 5. During a consultation, the user (patient) speaks, "I have been experiencing severe headaches for the past week."

[0174] 6. The device sends the voice data to the server, and the generative AI model translates it into Japanese as, "I've been experiencing a severe headache since last week."

[0175] 7. The terminal displays the translation results on the user's (medical professional's) screen.

[0176] 8. The user (medical professional) reviews the translation and asks any additional questions.

[0177] 9. The server encrypts all data and uses secure communication protocols.

[0178] 10. The generative model appropriately reflects cultural sensitivity where necessary.

[0179] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0180] Step 1:

[0181] A user accesses the system and enters the necessary information on the account registration screen, such as their name, contact information, and password. This input data is collected as the user's personal information.

[0182] Output: The entered user information is formatted as data.

[0183] Step 2:

[0184] The terminal sends the entered user information to the server, using a secure communication protocol such as SSL / TLS to ensure that the user can send data safely.

[0185] Output: The formatted user information is sent to the server as encrypted data.

[0186] Step 3:

[0187] The server stores the received user information in a database. A unique identifier (user ID) is generated for each user and authentication information is managed. This authentication information is used for subsequent logins.

[0188] Output: User information stored in the database and the generated user ID.

[0189] Step 4:

[0190] After logging in, the user selects the language to use on the language setting screen. They also specify whether they need medical translation. For example, they select English as their primary language and specify that they need translation.

[0191] Output: Selected language settings and translation needs.

[0192] Step 5:

[0193] The device sends the selected language preference information to the server, packaged in a format such as JSON and transmitted over a secure protocol.

[0194] Output: Language preference information is sent to the server.

[0195] Step 6:

[0196] The server loads the appropriate medical dictionary and medical database based on the received language setting information, for example, an English-Japanese medical dictionary.

[0197] Output: Loaded medical dictionary and database.

[0198] Step 7:

[0199] The server then integrates the loaded medical terminology dictionaries and databases into the generative AI model, passing the dictionary data as input to the model to ensure that it can translate specialized medical information.

[0200] Output: A generative AI model integrated with a medical terminology dictionary.

[0201] Step 8:

[0202] During the consultation, the user (patient) can input symptoms and questions by voice or text. For example, they can input "I have been experiencing severe headaches for the past week."

[0203] Output: Audio or text data.

[0204] Step 9:

[0205] The device sends the input voice and text data to the server. In the case of voice data, it is first converted into text using a speech recognition engine, for example, using the Google Cloud Speech-to-Text API.

[0206] Output: The audio data in text format is sent to the server.

[0207] Step 10:

[0208] The server uses a generative AI model to translate the received text data into the selected language (e.g., Japanese) in real time, for example, translating "I have been experiencing severe headaches for the past week" to "I have been experiencing severe headaches since last week."

[0209] Output: Real-time translated text data.

[0210] Step 11:

[0211] The device displays the translation results on the screen of the user (medical professional) and patient, and uses a front-end framework (such as React or Vue.js).

[0212] Output: The translation results are displayed on the screens of the medical professional and the patient.

[0213] Step 12:

[0214] The user (medical professional) checks the displayed translation and makes corrections or additions as necessary, for example adding the question "When did it start?"

[0215] Output: The modified or added text data.

[0216] Step 13:

[0217] The device then sends the corrected information back to the server, again packaged in JSON format or similar and transmitted over a secure protocol.

[0218] Output: The modified text data is sent to the server.

[0219] Step 14:

[0220] The server then uses the generative AI model to perform re-translation, translating the corrected text data back into the local language and providing accurate information to the user.

[0221] Output: The corrected translation text data.

[0222] Step 15:

[0223] The server encrypts all patient information and medical data and protects the data using secure communication protocols (SSL / TLS), specifically using encryption techniques such as AES.

[0224] Output: Encrypted user data and securely stored information.

[0225] Step 16:

[0226] The generative AI model applies training data provided by cultural experts to build algorithms that translate with cultural sensitivity, allowing translations to reflect cultural nuances, such as taking into account expressions and etiquette specific to Japanese culture.

[0227] Output: Culturally sensitive translation results.

[0228] (Application example 1)

[0229] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0230] When a language barrier exists between patients and medical professionals, it can be difficult for patients who speak different languages ​​or for the elderly to smoothly communicate and understand information in medical settings. Communication regarding the language settings of food menus is also a challenge. This can hinder smooth access to medical services and food delivery services.

[0231] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0232] In this invention, the server includes means for receiving a patient's language preference, means for translating medical information based on a generative model, means for displaying the translation result in real time, means for a medical professional to review and correct the translation result, means for protecting the medical information, means for applying a culturally sensitive algorithm, means for receiving a food menu's language preference, means for translating the food information based on a generative model, and means for displaying the translation result to restaurant staff, thereby enabling accurate communication of information between patients and food delivery service users in different language environments.

[0233] A "patient" is a person receiving medical services.

[0234] "Language settings" refers to settings that specify the language used by the user.

[0235] A "generative model" is an artificial intelligence that generates and translates text and speech based on large amounts of data and algorithms.

[0236] "Medical information" refers to all medical information, such as a patient's health condition, diagnosis results, and treatment plan.

[0237] A "translation result" is the result of a generative model converting input text or speech into another language.

[0238] "Display in real time" means that translated text or audio is displayed immediately.

[0239] "Healthcare professional" refers to a professional person engaged in healthcare, such as a doctor, nurse, or pharmacist.

[0240] "Protection" refers to measures taken to prevent unauthorized access or leakage of data.

[0241] A "culturally sensitive algorithm" is an algorithm that takes cultural background into account when producing translations and adjusts the content to be appropriate.

[0242] "Food Menu" means a list of foods and beverages that a Customer can order.

[0243] "Restaurant staff" refers to people who work in a restaurant and take orders and serve food.

[0244] This invention is a system that enables accurate communication between patients and food delivery service users, even in different language environments. The system receives the user's language settings, translates the information based on a generative model, and displays the translation results in real time.

[0245] Hardware and Software Configuration

[0246] The following hardware and software are used to implement this system:

[0247] Hardware: Smartphones, servers

[0248] Software: Firebase, Google Translate API, SpeechRecognition library

[0249] Details of data processing and data calculation

[0250] 1. A user uses a smartphone to input voice or text, such as when the user complains of a symptom, "I have been experiencing severe headaches," or when the user orders a dish, "I would like to order a spicy ramen."

[0251] 2. The device converts the voice data into text data using the SpeechRecognition library.

[0252] 3. The server receives the data sent from the device and translates it based on the generative model. The translation process uses the Google Translate API.

[0253] 4. The translation results are displayed in real time on the user's or staff's device. The displayed information is encrypted using a secure communication protocol to protect the data.

[0254] 5. The server applies a culturally sensitive algorithm and makes any necessary adjustments to the translation result, thereby providing a translation that is appropriate for the user's cultural background.

[0255] Specific example explanation

[0256] Scenario: English-speaking foreign patient

[0257] 1. The user (foreign patient) uses the smartphone app to set their language setting to "English."

[0258] 2. The device sends the registration information to the server, which loads the English-Japanese medical terminology dictionary.

[0259] 3. During a consultation, the user speaks, "I have been experiencing severe headaches."

[0260] 4. The device converts the voice data into text and sends it to the server.

[0261] 5. The server uses the generative model to translate the sentence into Japanese: "I've been experiencing severe headaches since last week."

[0262] 6. The device displays the translation results on the medical professional's screen, and the medical professional asks any additional questions.

[0263] Scenario: An English-speaking customer uses a Japanese food delivery service.

[0264] 1. The user accesses the app and sets the language setting to "English."

[0265] 2. The device sends the information to the server, which loads the English-Japanese menu database.

[0266] 3. The user says, "I would like to order a spicy ramen."

[0267] 4. The device converts the voice data into text and sends it to the server.

[0268] 5. The server uses the generative model to translate "I would like to order spicy ramen" into Japanese.

[0269] 6. The device displays the translation results on the restaurant staff's screen, and the staff asks questions to confirm the order.

[0270] Prompt Sentence Examples

[0271] "Translate the following text from English to Japanese: 'I would like to order a spicy ramen.'"

[0272] By using this system, patients who speak different languages ​​and food delivery service users can overcome communication barriers and receive services smoothly.

[0273] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0274] Step 1:

[0275] A user accesses the app using their smartphone and selects their language preference. As input, the user enters their language preference and sends it to the server. As output, the server receives the information and loads the appropriate terminology database.

[0276] Step 2:

[0277] The device receives voice data input from the user and converts it into text data. This process uses the SpeechRecognition library. The input is voice data, and the output is the converted text data.

[0278] Step 3:

[0279] The terminal sends the converted text data to the server. There is text data as input and data sent to the server as output.

[0280] Step 4:

[0281] The server translates the received text data using a generative model. This translation process uses the Google Translate API. The input is text data, and the output is the translated text.

[0282] Step 5:

[0283] The server sends the translated text to the user's and staff's terminals in real time. The input is the translation result, and the output is the translation result displayed on the terminal.

[0284] Step 6:

[0285] The user (medical professional or staff) can check the displayed translation result and make corrections or ask additional questions if necessary. The input is the translation result, and the output is the corrected information or additional questions.

[0286] Step 7:

[0287] The server encrypts medical information and food menu data and uses a secure communication protocol to protect the data. User and staff input data is used as input, and secure data communication is used as output.

[0288] Step 8:

[0289] The server applies a culturally sensitive algorithm to the generative model and adjusts the translation results as needed. The input is the translation results and cultural data, and the output is a culturally appropriate translation result.

[0290] Step 9:

[0291] The user (patient or consumer) can use their smartphone to check the translation results and facilitate the service (medical consultation or food order).The input is the adjusted translation result, and the output is the user's understanding and next action.

[0292] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0293] This system is designed to facilitate effective communication between patients and medical professionals, and is particularly designed to enable patients who speak different languages ​​and the elderly to smoothly communicate and understand information in medical settings. Furthermore, by incorporating an emotion engine that recognizes the user's emotions, this system allows the translation results to reflect the user's emotional state, enabling more appropriate and sensitive communication.

[0294] The program of this system mainly consists of the following steps:

[0295] User Registration and Authentication

[0296] 1. User: Accesses the system and enters name, contact information, password, etc. on the account registration screen.

[0297] 2. Terminal: Sends the entered registration information to the server.

[0298] 3. Server: Stores the submitted information in a database and generates authentication information that the user will use the next time they log in.

[0299] Verifying your language preferences and needs

[0300] 1. User: After logging in, select the language you want to use on the language setting screen. For example, select "English."

[0301] 2. Terminal: Sends the selected language setting information to the server.

[0302] 3. Server: Based on the received language settings, loads the appropriate medical dictionary and medical database.

[0303] Loading the medical terminology database

[0304] 1. Server: Loads appropriate medical dictionaries and medical databases and integrates them into the generative model, enabling professional medical information translation.

[0305] Real-time conversation translation

[0306] 1. User (Patient): During the consultation, the patient enters symptoms and questions by voice or text (e.g., "I have been experiencing severe headaches for the past week").

[0307] 2. Terminal: Sends input voice and text data to the server.

[0308] 3. Server: The generative model translates the received data into the specified language (e.g., Japanese) (e.g., "I have been experiencing severe headaches since last week").

[0309] 4. Terminal: The translation results are displayed in real time on the screens of the user (medical professional) and the patient.

[0310] Check and complete translation results

[0311] 1. User (medical professional): Checks the displayed translation and makes corrections or completions as necessary, for example, checking for additional symptoms.

[0312] 2. Terminal: Send the corrected information back to the server.

[0313] 3. Server: The generative model re-translates the corrected data and generates the final translation result.

[0314] Information Protection and Security

[0315] 1. Server: All patient information and medical data is encrypted and protected using secure communication protocols.

[0316] 2. Device: Configured to ensure secure data communications.

[0317] Cultural sensitivity

[0318] 1. Server: Implements the algorithm that applies training data from cultural experts to the generative model and produces culturally sensitive translations.

[0319] 2. Generative model: Reflects cultural considerations when they are necessary for the translation results.

[0320] Emotion recognition

[0321] 1. Users: When describing symptoms or questions by voice or text, they may also express emotions (e.g., "I'm very worried about my severe headaches").

[0322] 2. Terminal: Sends emotion data along with voice and text data to the server.

[0323] 3. Server: The emotion engine recognizes the user's emotional state from speech and text and applies the analysis results to the generative model.

[0324] 4. Generative model: Appropriately reflects emotional information in the translation results, producing translations that convey more emotional nuance.

[0325] Specific examples

[0326] Scenario: An English-speaking foreign patient visits a Japanese hospital

[0327] 1. The user (patient) accesses the system and registers as a user.

[0328] 2. The device sends the registration information to the server, which stores the information in a database.

[0329] 3. The user (patient) selects "English" in the language settings, and the device sends that information to the server.

[0330] 4. The server loads the English-Japanese medical terminology dictionary and integrates it into the generative model.

[0331] 5. During a consultation, the user (patient) speaks, "I have been experiencing severe headaches for the past week. I'm very worried about it."

[0332] 6. The device sends the emotion data along with the voice data to the server, and the emotion engine analyzes the emotion "worried."

[0333] 7. Based on the data, the server translates the sentence into Japanese as "I have been experiencing severe headaches since last week and I am very worried about it."

[0334] 8. The terminal displays the translation results on the user's (medical professional's) screen.

[0335] 9. The user (medical professional) reviews the translation results and asks any additional questions or clarifications.

[0336] 10. The server encrypts all data and uses secure communication protocols.

[0337] 11. The generative model appropriately reflects cultural sensitivity where necessary.

[0338] This system will enable elderly and foreign patients to receive medical care smoothly and ensure the accuracy and reliability of medical information. In addition, by combining it with an emotion engine, more detailed communication will be possible.

[0339] The processing flow will be explained below.

[0340] Step 1:

[0341] User: Accesses the system and enters name, contact information, password, etc. on the account registration screen.

[0342] Step 2:

[0343] Terminal: Sends the entered registration information to the server.

[0344] Step 3:

[0345] Server: Stores the submitted information in a database and generates authentication information that the user will use the next time they log in.

[0346] Step 4:

[0347] User: After logging in, select the language to use on the language setting screen. For example, select "English."

[0348] Step 5:

[0349] Terminal: Sends the selected language setting information to the server.

[0350] Step 6:

[0351] Server: Loads the appropriate medical dictionary and medical database based on the selected language settings.

[0352] Step 7:

[0353] Server: Prepares for translation by integrating the loaded medical dictionary and medical database into the generative model.

[0354] Step 8:

[0355] User (Patient): During the consultation, the patient will voice or text their symptoms and questions (e.g., "I have been experiencing severe headaches for the past week. I'm very worried about it").

[0356] Step 9:

[0357] Terminal: Sends input voice and text data to the server.

[0358] Step 10:

[0359] Server: The emotion engine recognizes the emotion "worried" from speech and text, and applies the analysis results to the generative model.

[0360] Step 11:

[0361] Server: The generative model translates the received data into the specified language (e.g., Japanese) (e.g., "I've been experiencing severe headaches since last week. I'm very worried about it").

[0362] Step 12:

[0363] Terminal: Displays the translation results in real time on the user's (medical professional's) screen.

[0364] Step 13:

[0365] User (medical professional): Checks the displayed translation and makes corrections or completions as necessary, for example, checking for additional symptoms.

[0366] Step 14:

[0367] Terminal: Send the corrected information back to the server.

[0368] Step 15:

[0369] Server: The generative model re-translates the corrected data and generates the final translation result.

[0370] Step 16:

[0371] Terminal: displays the final translation results on the screen of the user (patient) and medical professional.

[0372] Step 17:

[0373] Server: Protects patient personal information and medical data by encrypting it and applying secure communication protocols.

[0374] Step 18:

[0375] Server: Implements the algorithm that applies training data provided by cultural experts to the generative model and performs translation with cultural sensitivity.

[0376] Step 19:

[0377] Generative model: Properly reflecting the content of translations that require cultural sensitivity.

[0378] In this way, the system of the present invention creates an environment where elderly and foreign patients can receive medical care smoothly, ensuring the accuracy and reliability of medical information. In addition, by combining it with an emotion engine, more detailed communication is possible.

[0379] Example 2

[0380] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0381] In today's medical environment, patients who speak different languages ​​and the elderly face difficulties in communicating appropriately in medical settings. Furthermore, simple translation without considering the patient's emotional state makes it difficult for medical professionals to understand the patient's true needs and feelings. Furthermore, confidentiality of medical information and cultural sensitivity are also important, but there is still a lack of systems that can adequately address these issues.

[0382] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0383] In this invention, the server includes means for receiving the patient's language setting, means for recognizing the patient's emotional state, means for translating medical information based on a generative model, means for displaying the translation results in real time, means for a medical professional to review and correct the translation results, means for protecting the medical information, and means for applying a culturally sensitive algorithm, thereby enabling smooth communication between patients and elderly people who speak different languages ​​in medical settings and realizing sensitive and accurate medical information transmission that takes into account the patient's emotional state and cultural background.

[0384] The "means for receiving language settings" is a function that allows the user to specify the language they want to use and transmit that information to the server.

[0385] The "means for recognizing emotional states" is a function that analyzes and identifies emotions contained in the user's statements and inputs.

[0386] "Means for translating medical information based on a generative model" refers to a function that uses a generative AI model to translate user utterances or input into a specified language.

[0387] "Means for displaying translation results in real time" refers to a function that immediately outputs the translated content to a display device.

[0388] "Means for medical professionals to check and correct translation results" is a function that allows medical professionals to check the displayed translation results and correct them if necessary.

[0389] "Measures for protecting medical information" refers to the ability to encrypt patient information and medical data and protect the confidentiality of data using secure communication protocols.

[0390] "Means for applying a culturally sensitive algorithm" refers to a function that applies an algorithm that takes cultural background into account and makes appropriate adjustments to the translation results.

[0391] The present invention provides a system that enables patients and elderly people who speak different languages ​​to smoothly communicate and understand information in medical settings. The system includes a means for receiving a patient's language preference, a means for recognizing an emotional state, a means for translating medical information based on a generative model, a means for displaying the translation results in real time, a means for a medical professional to review and correct the translation results, a means for protecting medical information, and a means for applying a culturally sensitive algorithm.

[0392] User Registration and Authentication

[0393] 1. The user accesses the system and enters their name, contact information, and password on the account registration screen. The device sends the entered registration information to the server. The server stores the received information in a database and generates a user ID and authentication token. This authentication token is used the next time the user logs in.

[0394] Verifying your language preferences and needs

[0395] 1. After logging in, the user selects the language they want to use on the language setting screen. For example, they select "English." The device sends this language setting information to the server. Based on the selected language, the server loads the appropriate medical terminology dictionary and medical database and integrates them into the generative model.

[0396] Loading the medical terminology database

[0397] 1. The server integrates medical terminology dictionaries and medical databases into the generative model to enable the translation of specialized medical information. This process ensures accurate translation of specialized terminology and medical-related information.

[0398] Real-time conversation translation

[0399] 1. During a consultation, the user (patient) inputs their symptoms or questions by voice or text. For example, they might input "I have been experiencing severe headaches for the past week." The device sends the input voice or text data to the server. The server uses a generative model to translate the received data into the specified language (for example, Japanese). The translation result is "I have been experiencing severe headaches since last week." The device displays the translation result in real time on the screens of the user (medical professional) and the patient.

[0400] Check and complete translation results

[0401] 1. The user (medical professional) checks the displayed translation results and makes corrections or additions as necessary. The device then sends the corrected information back to the server, where the generative model re-translates the corrected data to generate the final translation results.

[0402] Information Protection and Security

[0403] 1. The server encrypts all patient information and medical data and protects the data using secure communication protocols. The device is configured to ensure secure data communication.

[0404] Cultural sensitivity

[0405] 1. The server applies training data provided by cultural experts to a generative model, implementing an algorithm that achieves culturally sensitive translation. The generative model then reflects any cultural considerations that may be required in the translation results.

[0406] Emotion recognition

[0407] 1. When a user inputs symptoms or questions using voice or text, they also express their emotions. For example, they might input "I'm very worried about my severe headaches." The device sends the emotion data along with the voice and text data to the server. The server's emotion engine analyzes the user's emotional state from the voice or text and applies the analysis results to a generative model. The generative model then incorporates the emotion information into the translation results, resulting in a translation that conveys more emotional nuance.

[0408] Specific examples

[0409] Scenario: An English-speaking foreign patient visits a Japanese hospital

[0410] 1. The user (patient) accesses the system, enters their name, contact information, and password on the registration screen, enters the required information in the form, and clicks the submit button.

[0411] 2. The device sends the registration information to the server, after which a registration success message is displayed.

[0412] 3. The server saves the information to a database and generates an authentication token. The user ID and token are registered in the database.

[0413] 4. The user (patient) logs in and selects "English" in the language settings. Select English from the drop-down menu and click the Save button.

[0414] 5. The device sends the language setting information to the server.

[0415] 6. The server loads the English-Japanese medical terminology dictionary and integrates it into the generative model.

[0416] 7. During a consultation, the user (patient) dictates, "I have been experiencing severe headaches for the past week. I'm very worried about it." This can be done using a dedicated microphone or a text input field.

[0417] 8. The device sends the voice data and emotion data to the server, and the emotion engine analyzes the emotion "worried."

[0418] 9. Based on the data, the server translates the sentence into Japanese: "I have been experiencing severe headaches since last week and I am very worried about it."

[0419] 10. The device displays the translation results on the screen of the user (medical professional) and patient. The translation results are displayed in real time on the screen of a smartphone or tablet.

[0420] 11. The user (medical professional) checks the translation results and asks any additional questions or clarifications. Checks the content displayed on the tablet and enters additional instructions if necessary.

[0421] 12. The server encrypts all data and uses secure communication protocols.

[0422] 13. The generative model appropriately reflects cultural sensitivity where necessary.

[0423] Specific prompt examples

[0424] "In a scenario where an English-speaking foreign patient is seen at a hospital in Japan, and the patient says, 'I have been experiencing severe headaches for the past week. I'm very worried about it,' how would this system work?"

[0425] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0426] Step 1:

[0427] A user accesses the system and enters their name, contact information, and password on the account registration screen. The device sends the entered registration information to the server. The server stores the received information in a database and generates a user ID and authentication token.

[0428] Input: Name, Contact, Password

[0429] Output: User ID, authentication token

[0430] Specific operation: The user enters information into the form and clicks the submit button. The device sends the information to the server, which stores it in the database.

[0431] Step 2:

[0432] The user logs in and selects the language to use on the language setting screen. The terminal sends the language setting information to the server. The server loads the medical terminology dictionary and medical database based on the selected language.

[0433] Input: Login information, language setting information

[0434] Output: Medical terminology dictionary, medical database

[0435] Specific operation: A user logs in, selects the language they want to use from a drop-down menu, and clicks the Save button. The device sends the language setting information to the server, which loads the appropriate database.

[0436] Step 3:

[0437] The server integrates medical terminology dictionaries and medical databases into the generative model, preparing it for the translation of specialized medical information.

[0438] Input: Medical terminology dictionary, medical database

[0439] Output: A unified generative model

[0440] Specific operation: The server loads the medical terminology dictionary and medical database and performs the process of integrating them into the generative model.

[0441] Step 4:

[0442] During a consultation, the user (patient) inputs symptoms and questions by voice or text. The device sends the input voice or text data to the server, where a generative model translates the data and converts it into the specified language.

[0443] Input: Audio data, text data

[0444] Output: Translation data

[0445] Specific operation: The user speaks into the microphone or types into the text field. The device sends the data to the server, which then translates it using the generative model.

[0446] Step 5:

[0447] The terminal displays the translation results in real time on the screen of the user (medical professional) and the patient.

[0448] Input: Translation data

[0449] Output: Translation results displayed on the screen

[0450] Specific operation: The translation results are displayed in real time on the screen of your smartphone or tablet.

[0451] Step 6:

[0452] The user (medical professional) checks the displayed translation results and makes corrections or additions as necessary. The device then sends the corrected information back to the server, where the generative model re-translates the corrected data to generate the final translation results.

[0453] Input: Corrected translation data

[0454] Output: Final translation data

[0455] How it works: A medical professional reviews the translation results, enters corrections on the tablet, and the device sends the data back to the server, which then uses the generative model to re-translate the corrected data.

[0456] Step 7:

[0457] The server encrypts all patient information and medical data and protects the data using secure communication protocols.

[0458] Input: Patient information, medical data

[0459] Output: Encrypted data, secure communication

[0460] What happens: The server encrypts the data using an encryption algorithm and establishes a secure communication protocol.

[0461] Step 8:

[0462] The server implements an algorithm that applies training data provided by cultural experts to a generative model to achieve culturally sensitive translation.

[0463] Input: Training data

[0464] Output: Culturally sensitive algorithm

[0465] Specific operation: The server applies the training data to the generative model and updates the algorithm.

[0466] Step 9:

[0467] When users input symptoms or questions by voice or text, they also express their emotions. The device sends the emotion data along with the voice and text data to the server. The server's emotion engine analyzes the user's emotional state from the voice or text and applies the analysis results to a generative model. The generative model reflects the emotion information in the translation results, resulting in a translation that conveys more emotional nuance.

[0468] Input: Voice data, text data, emotion data

[0469] Output: Translation data reflecting emotional information

[0470] How it works: The user inputs their emotions through voice or text. The device sends the data to the server, which then uses an emotion engine to recognize the emotional state and apply it to the generative model.

[0471] (Application example 2)

[0472] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0473] The problem that this invention aims to solve is to support patients and elderly people who speak different languages ​​in smoothly communicating and understanding information in medical settings, and to realize more appropriate communication through emotion recognition. Another problem is to achieve both data protection and cultural considerations. Furthermore, it is expected that the same effect will be achieved in virtual stores as an application.

[0474] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0475] In this invention, the server includes means for receiving a patient's language setting, means for translating medical information based on a generative model, means for displaying the translation result in real time, means for a medical professional to check and correct the translation result, means for protecting the medical information, means for applying a culturally sensitive algorithm, and means for recognizing emotional states using an emotion engine and reflecting them in the translation result, thereby enabling smooth and appropriate communication between patients, elderly people, and even virtual store customers who speak different languages.

[0476] The "means for receiving patient language setting" is an interface for the patient to select or set the language they use and send that information to the system.

[0477] "Means for translating medical information based on a generative model" means a function that uses a generative AI model to translate specialized medical information into different languages ​​based on received language preferences.

[0478] "Means for displaying translation results in real time" refers to a display or user interface that instantly displays translated medical information so that patients and medical professionals can quickly review it.

[0479] "Means for medical professionals to review and correct translation results" refers to a function that allows medical professionals to review the translation results displayed by the system and correct or supplement them as necessary.

[0480] "Measures to protect medical information" refers to the use of encryption and secure communication protocols to protect patient privacy and safely handle medical information.

[0481] "Means for applying culturally sensitive algorithms" are algorithms that use training data that takes into account cross-cultural sensitivities and that reflect cultural context in translated medical information.

[0482] "Means of recognizing emotional states using an emotion engine and reflecting them in the translation results" is a function that analyzes the emotions contained in the patient's voice or text and appropriately reflects that emotional state in the translation results.

[0483] The present invention is implemented as a system for facilitating communication between patients and medical professionals who speak different languages, and can also support communication between customers and staff in virtual stores.

[0484] The system mainly consists of a server, a terminal, and a user. In this embodiment, the system is implemented using terminals such as smartphones, smart glasses, and head-mounted displays, and uses generative AI models and related algorithms on the server.

[0485] Overall system flow

[0486] 1. User Registration and Authentication

[0487] The user accesses the system and registers an account by entering their name, contact information, password, etc.

[0488] The device sends the entered information to the server, which stores it in a database and generates authentication information to use for subsequent logins.

[0489] 2. Identifying language preferences and needs

[0490] After logging in, the user sets the language to be used, for example, English or Japanese.

[0491] The terminal transmits the selected language information to the server, which then loads the corresponding medical terminology dictionary and product database.

[0492] 3. Real-time translation of conversations

[0493] Users (patients or customers) input questions or symptoms by voice or text. For example, a voice input might say, "I have been experiencing severe headaches for the past week."

[0494] The device sends the input data to the server, which then translates it using a generative AI model (using TensorFlow), and displays the translation results on the device in real time.

[0495] In addition, an emotion engine (using the Azure Emotion API) analyzes emotions and reflects them in the translation results.

[0496] 4. Check and complete the translation result

[0497] The user (medical professional or staff) reviews the translation and corrects it if necessary.

[0498] The device then sends the corrected information back to the server, and the final translation result is generated.

[0499] 5. Data Protection and Security

[0500] The server encrypts all data and protects it with secure communication protocols (SSL / TLS).

[0501] This allows for smooth and appropriate communication with patients and customers who speak different languages, and the system is also culturally sensitive, supporting more natural communication.

[0502] Specific examples

[0503] Use in medical settings

[0504] Patient (English speaker): "I have been experiencing severe headaches for the past week. I'm very worried about it."

[0505] The device sends voice and emotion data to the server, which translates it into "I've been experiencing severe headaches since last week and I'm very worried about it," and displays the results reflecting the emotion.

[0506] Use in virtual stores

[0507] Customer (English speaker): "What ingredients are in this skincare product? I have sensitive skin and I'm really worried about potential reactions."

[0508] The device sends voice and emotion data to the server, which translates it into "What ingredients are in this skin care product? I have sensitive skin, so I'm worried about a reaction," and displays the results reflecting the emotion.

[0509] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0510] Step 1:

[0511] A user accesses the system and enters their name, contact information, password, etc. on the account registration screen. The device sends this information to the server, which then stores it in a database. Authentication information to be used for subsequent logins is also generated.

[0512] Input: Name, Contact, Password

[0513] Data processing: Save the input information in the database and generate authentication information

[0514] Output: Credentials

[0515] Step 2:

[0516] After logging in, the user selects the language they want to use. The terminal sends the selected language information to the server, which then loads the corresponding medical terminology dictionary and product database based on that information, thereby preparing the necessary translation resources.

[0517] Input: Select your language

[0518] Data processing: Load medical dictionary and product database based on language settings

[0519] Output: Available translation resources

[0520] Step 3:

[0521] The user (patient or customer) inputs a question or symptom via voice or text. For example, a voice input might say, "I have been experiencing severe headaches for the past week." The device sends this to the server, which then uses a generative AI model to translate the input data.

[0522] Input: Voice or text input

[0523] Data transformation: Translating input data using generative AI models

[0524] Output: Translation result

[0525] Step 4:

[0526] The device receives the voice and text data sent to the server and uses an emotion engine to recognize the emotional state. Using this emotion data, the server reflects the emotional information in the translation results.

[0527] Input: Voice and text data

[0528] Data processing: Emotion recognition by emotion engine and reflection in translation results

[0529] Output: Translation results that reflect emotional information

[0530] Step 5:

[0531] The translation results are displayed on the device in real time. Users (medical professionals and staff) can review the translation results and make corrections as needed. The corrected information is then sent back to the server, where the final translation results are generated.

[0532] Input: Translation results that reflect emotional information

[0533] Data processing: User checks and corrects, and final translation results are generated.

[0534] Output: The final translation result

[0535] Step 6:

[0536] All data is encrypted and protected using secure communication protocols (SSL / TLS), ensuring patient and customer information is kept secure.

[0537] Input: Translation results and user information

[0538] Data processing: Encrypting data and using secure communication protocols

[0539] Output: Secure data

[0540] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0541] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0542] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0543] [Second embodiment]

[0544] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0545] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0546] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0547] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0548] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0549] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0550] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0551] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0552] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0553] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0554] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0555] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0556] The present invention is a system for effective communication between patients and medical professionals, and is designed to enable patients who speak different languages ​​and the elderly to smoothly communicate and understand information in medical settings.

[0557] The program of this system mainly consists of the following steps:

[0558] User Registration and Authentication

[0559] 1. User: Accesses the system and enters the required information (name, contact information, password, etc.) on the account registration screen.

[0560] 2. Terminal: Sends the entered user information to the server.

[0561] 3. Server: Stores user information in a database and generates and manages authentication information, which is used for subsequent logins.

[0562] Verifying your language preferences and needs

[0563] 1. User: After logging in, select the language you want to use on the language settings screen. Also, specify whether you need medical translation.

[0564] 2. Terminal: Sends the selected language setting information to the server.

[0565] 3. Server: Based on the received language settings, loads the appropriate medical dictionary and medical database.

[0566] Loading the medical terminology database

[0567] 1. Server: Loads appropriate medical dictionaries and medical databases and integrates them into the generative model, enabling professional medical information translation.

[0568] Real-time conversation translation

[0569] 1. User (Patient): During a consultation, the patient enters symptoms or questions via voice or text (e.g., "I've had a bad headache lately").

[0570] 2. Terminal: Sends input voice and text data to the server.

[0571] 3. Server: The generative model processes the received data and translates it in real time into the selected language (e.g., "I have been experiencing severe headaches recently").

[0572] 4. Terminal: Displays the translation results on the screen of the user (medical professional) and patient.

[0573] Check and complete translation results

[0574] 1. User (medical professional): Check the displayed translation results and make corrections or additions as necessary. This allows for correction of mistranslations and missing information.

[0575] 2. Terminal: Send the corrected information back to the server.

[0576] 3. Server: The generative model retranslates the corrected information and provides it to patients and specialists.

[0577] Information Protection and Security

[0578] 1. Server: All patient information and medical data is encrypted and protected using secure communication protocols.

[0579] 2. Device: Configured to ensure secure data communications.

[0580] Cultural sensitivity

[0581] 1. Server: Implements the algorithm that applies training data from cultural experts to the generative model and produces culturally sensitive translations.

[0582] 2. Generative model: Reflects cultural considerations when they are necessary for the translation results.

[0583] Specific examples

[0584] Scenario: An English-speaking foreign patient visits a Japanese hospital

[0585] 1. The user (patient) accesses the system and registers as a user.

[0586] 2. The device sends the registration information to the server, which stores the information in a database.

[0587] 3. The user (patient) selects "English" in the language settings, and the device sends that information to the server.

[0588] 4. The server loads the English-Japanese medical terminology dictionary and integrates it into the generative model.

[0589] 5. During a consultation, the user (patient) speaks, "I have been experiencing severe headaches for the past week."

[0590] 6. The device sends the voice data to the server, and the generative model translates it into Japanese as "I've been experiencing a severe headache since last week."

[0591] 7. The terminal displays the translation results on the user's (medical professional's) screen.

[0592] 8. The user (medical professional) reviews the translation and asks any additional questions.

[0593] 9. The server encrypts all data and uses secure communication protocols.

[0594] 10. The generative model appropriately reflects cultural sensitivity where necessary.

[0595] This system will enable elderly and foreign patients to receive medical care smoothly and ensure the accuracy and reliability of medical information.

[0596] The processing flow will be explained below.

[0597] Step 1:

[0598] User: Accesses the system and enters name, contact information, password, etc. on the account registration screen.

[0599] Step 2:

[0600] Terminal: Sends the entered registration information to the server.

[0601] Step 3:

[0602] Server: Stores the submitted information in a database and generates authentication information that the user will use the next time they log in.

[0603] Step 4:

[0604] User: After logging in, select the language to use on the language setting screen. For example, select "English."

[0605] Step 5:

[0606] Terminal: Sends the selected language setting information to the server.

[0607] Step 6:

[0608] Server: Loads the appropriate medical dictionary and medical database based on the selected language settings.

[0609] Step 7:

[0610] Server: Prepares for translation by integrating the loaded medical dictionary and medical database into the generative model.

[0611] Step 8:

[0612] User (Patient): During the consultation, the patient will voice or text in their symptoms and questions (e.g., "I have been experiencing severe headaches for the past week").

[0613] Step 9:

[0614] Terminal: Sends input voice and text data to the server.

[0615] Step 10:

[0616] Server: The generative model translates the received data into the specified language (e.g., Japanese) (e.g., "I have been experiencing severe headaches since last week").

[0617] Step 11:

[0618] Terminal: Displays the translation results in real time on the user's (medical professional's) screen.

[0619] Step 12:

[0620] User (medical professional): Checks the displayed translation and makes corrections or completions as necessary, for example, checking for additional symptoms.

[0621] Step 13:

[0622] Terminal: Send the corrected information back to the server.

[0623] Step 14:

[0624] Server: The generative model re-translates the corrected data and generates the final translation result.

[0625] Step 15:

[0626] Terminal: displays the final translation results on the screen of the user (patient) and medical professional.

[0627] Step 16:

[0628] Server: Protects patient personal information and medical data by encrypting it and applying secure communication protocols.

[0629] Step 17:

[0630] Server: Implements the algorithm that applies training data provided by cultural experts to the generative model and performs translation with cultural sensitivity.

[0631] Step 18:

[0632] Generative model: Properly reflecting the content of translations that require cultural sensitivity.

[0633] Through the above steps, the system of the present invention creates an environment in which elderly and foreign patients can receive medical care smoothly, and ensures the accuracy and reliability of medical information.

[0634] Example 1

[0635] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0636] Patients and the elderly who speak different languages ​​often have difficulty communicating and understanding information smoothly in medical settings. In particular, inaccurate medical terminology and explanations of symptoms can hinder diagnosis and treatment. Furthermore, translation errors and a lack of cultural sensitivity can lead to misunderstandings between patients and medical professionals.

[0637] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0638] In this invention, the server includes means for receiving a patient's language setting, means for translating medical information based on a generative AI model, means for displaying the translation results in real time, means for medical professionals to review and correct the translation results, means for protecting the medical information, means for applying a culturally sensitive algorithm, means for converting voice data to text, means for loading a medical terminology dictionary and a medical database, and means for smooth information exchange between patients and medical professionals. This enables patients and elderly people who speak different languages ​​to exchange information accurately and quickly in medical settings, improving the accuracy and reliability of medical information transmission.

[0639] The "means for receiving patient language setting" is an interface through which the system inputs and acquires information specifying the language used by the patient.

[0640] A "means for translating medical information based on a generative AI model" is a function that uses an artificial intelligence model to translate input medical-related information into different languages.

[0641] "Means for displaying the translation results in real time" refers to a technique for instantly displaying the translated information on the user's display device.

[0642] "Means for medical professionals to review and correct translation results" refers to an interface that allows medical professionals to review the translation results provided by the system and make corrections or additions as necessary.

[0643] "Measures to protect medical information" are encryption technologies and security protocols used to protect medical information, such as patient personal information and medical records, from third parties.

[0644] "Means for applying culturally sensitive algorithms" are algorithms that allow the generative AI model to take cultural background and specific context into account when translating.

[0645] "Means for converting voice data to text" refers to speech recognition technology that analyzes voice input and provides the content to the system as text data.

[0646] The "means for loading medical terminology dictionaries and medical databases" refers to a function for loading dictionaries and databases containing medical terminology and related information into the system.

[0647] "Means for smooth information exchange between patients and medical professionals" refers to interfaces and communication technologies that allow patients and medical professionals to organize information and communicate it quickly and accurately.

[0648] The present invention relates to a system for effective communication between patients and medical professionals. It is designed to enable patients who speak different languages ​​and elderly people to smoothly communicate and understand information in medical settings. The system is realized using the following main hardware and software:

[0649] Hardware and software used

[0650] Server: A high-performance backend server for handling databases, generative AI models, and encryption technologies.

[0651] Device: Computers and smart devices (e.g., PCs, tablets, smartphones) used by patients and healthcare professionals.

[0652] Generative AI models: Use OpenAI GPT-4 or similar language models to translate medical information.

[0653] Speech recognition engine: A technology that converts speech to text, such as the Google Cloud Speech-to-Text API.

[0654] Database: A relational database such as MySQL or PostgreSQL to manage user information and a medical dictionary.

[0655] Front-end technology: Frameworks for creating user interfaces, such as React and Vue.js.

[0656] Program processing flow

[0657] The system consists of the following steps:

[0658] 1. User Registration and Authentication

[0659] 1. The user (patient) accesses the system and enters the required information such as name, contact information, password, etc. The entered information is sent to the server using a secure communication protocol (SSL / TLS).

[0660] 2. The server stores the received user information in a database and generates a unique identifier (user ID).

[0661] 3. The server generates authentication information and manages the user account. This authentication information is used for subsequent logins.

[0662] 2. Identifying language preferences and needs

[0663] 1. After logging in, the user selects the language they want to use on the language settings screen. For example, they can choose English as their primary language. They can also specify whether they need medical translation.

[0664] 2. The terminal sends the selected language setting information to the server, and the server loads the appropriate medical terminology dictionary and medical database based on that information.

[0665] 3. Loading the Medical Terminology Database

[0666] 1. The server loads an open-source medical dictionary (e.g., MedDRA or SNOMED CT) according to the user's language settings.

[0667] 2. The server integrates the loaded dictionaries and databases into a generative AI model, enabling the translation of specialized medical information.

[0668] 4. Real-time translation of conversations

[0669] 1. The user (patient) enters symptoms and questions by voice or text during a consultation. For example, the user might enter "I have been experiencing severe headaches for the past week."

[0670] 2. The device sends the voice data to the server, which uses a speech recognition engine to convert the voice into text.

[0671] 3. The server uses the generative AI model to translate the text data into the selected language (e.g., Japanese) in real time.

[0672] 4. The terminal displays the translation result on the screen of the user (medical professional) and the patient. For example, the translation may read, "I have been experiencing a severe headache since last week."

[0673] 5. Check and complete the translation result

[0674] 1. The user (medical professional) checks the displayed translation and makes corrections or additions as necessary. For example, adding an additional question such as "When did it start?"

[0675] 2. The terminal sends the corrected information back to the server, and the server performs a retranslation.

[0676] 6. Data Protection and Security

[0677] 1. The server encrypts all patient information and medical data using standard encryption techniques such as AES.

[0678] 2. The server protects data using a secure communication protocol (SSL / TLS) and achieves encrypted communication using public and private keys.

[0679] 3. The device is configured to ensure secure data communications.

[0680] 7. Cultural Sensitivity

[0681] 1. The server applies training data from cultural experts to a generative AI model, implementing an algorithm that produces translations that reflect cultural nuances.

[0682] 2. The generative model will incorporate cultural considerations where necessary, for example, translating with consideration for expressions and etiquette specific to Japanese culture.

[0683] Specific examples

[0684] Scenario: An English-speaking foreign patient visits a Japanese hospital

[0685] 1. The user (patient) accesses the system and registers as a user.

[0686] 2. The device sends the registration information to the server, which stores the information in a database.

[0687] 3. The user (patient) selects "English" in the language settings, and the device sends that information to the server.

[0688] 4. The server loads an English-Japanese medical terminology dictionary and integrates it into the generative AI model.

[0689] 5. During a consultation, the user (patient) speaks, "I have been experiencing severe headaches for the past week."

[0690] 6. The device sends the voice data to the server, and the generative AI model translates it into Japanese as, "I've been experiencing a severe headache since last week."

[0691] 7. The terminal displays the translation results on the user's (medical professional's) screen.

[0692] 8. The user (medical professional) reviews the translation and asks any additional questions.

[0693] 9. The server encrypts all data and uses secure communication protocols.

[0694] 10. The generative model appropriately reflects cultural sensitivity where necessary.

[0695] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0696] Step 1:

[0697] A user accesses the system and enters the necessary information on the account registration screen, such as their name, contact information, and password. This input data is collected as the user's personal information.

[0698] Output: The entered user information is formatted as data.

[0699] Step 2:

[0700] The terminal sends the entered user information to the server, using a secure communication protocol such as SSL / TLS to ensure that the user can send data safely.

[0701] Output: The formatted user information is sent to the server as encrypted data.

[0702] Step 3:

[0703] The server stores the received user information in a database. A unique identifier (user ID) is generated for each user and authentication information is managed. This authentication information is used for subsequent logins.

[0704] Output: User information stored in the database and the generated user ID.

[0705] Step 4:

[0706] After logging in, the user selects the language to use on the language setting screen. They also specify whether they need medical translation. For example, they select English as their primary language and specify that they need translation.

[0707] Output: Selected language settings and translation needs.

[0708] Step 5:

[0709] The device sends the selected language preference information to the server, packaged in a format such as JSON and transmitted over a secure protocol.

[0710] Output: Language preference information is sent to the server.

[0711] Step 6:

[0712] The server loads the appropriate medical dictionary and medical database based on the received language setting information, for example, an English-Japanese medical dictionary.

[0713] Output: Loaded medical dictionary and database.

[0714] Step 7:

[0715] The server then integrates the loaded medical terminology dictionaries and databases into the generative AI model, passing the dictionary data as input to the model to ensure that it can translate specialized medical information.

[0716] Output: A generative AI model integrated with a medical terminology dictionary.

[0717] Step 8:

[0718] During the consultation, the user (patient) can input symptoms and questions by voice or text. For example, they can input "I have been experiencing severe headaches for the past week."

[0719] Output: Audio or text data.

[0720] Step 9:

[0721] The device sends the input voice and text data to the server. In the case of voice data, it is first converted into text using a speech recognition engine, for example, using the Google Cloud Speech-to-Text API.

[0722] Output: The audio data in text format is sent to the server.

[0723] Step 10:

[0724] The server uses a generative AI model to translate the received text data into the selected language (e.g., Japanese) in real time, for example, translating "I have been experiencing severe headaches for the past week" to "I have been experiencing severe headaches since last week."

[0725] Output: Real-time translated text data.

[0726] Step 11:

[0727] The device displays the translation results on the screen of the user (medical professional) and patient, and uses a front-end framework (such as React or Vue.js).

[0728] Output: The translation results are displayed on the screens of the medical professional and the patient.

[0729] Step 12:

[0730] The user (medical professional) checks the displayed translation and makes corrections or additions as necessary, for example adding the question "When did it start?"

[0731] Output: The modified or added text data.

[0732] Step 13:

[0733] The device then sends the corrected information back to the server, again packaged in JSON format or similar and transmitted over a secure protocol.

[0734] Output: The modified text data is sent to the server.

[0735] Step 14:

[0736] The server then uses the generative AI model to perform re-translation, translating the corrected text data back into the local language and providing accurate information to the user.

[0737] Output: The corrected translation text data.

[0738] Step 15:

[0739] The server encrypts all patient information and medical data and protects the data using secure communication protocols (SSL / TLS), specifically using encryption techniques such as AES.

[0740] Output: Encrypted user data and securely stored information.

[0741] Step 16:

[0742] The generative AI model applies training data provided by cultural experts to build algorithms that translate with cultural sensitivity, allowing translations to reflect cultural nuances, such as taking into account expressions and etiquette specific to Japanese culture.

[0743] Output: Culturally sensitive translation results.

[0744] (Application example 1)

[0745] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0746] When a language barrier exists between patients and medical professionals, it can be difficult for patients who speak different languages ​​or for the elderly to smoothly communicate and understand information in medical settings. Communication regarding the language settings of food menus is also a challenge. This can hinder smooth access to medical services and food delivery services.

[0747] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0748] In this invention, the server includes means for receiving a patient's language preference, means for translating medical information based on a generative model, means for displaying the translation result in real time, means for a medical professional to review and correct the translation result, means for protecting the medical information, means for applying a culturally sensitive algorithm, means for receiving a food menu's language preference, means for translating the food information based on a generative model, and means for displaying the translation result to restaurant staff, thereby enabling accurate communication of information between patients and food delivery service users in different language environments.

[0749] A "patient" is a person receiving medical services.

[0750] "Language settings" refers to settings that specify the language used by the user.

[0751] A "generative model" is an artificial intelligence that generates and translates text and speech based on large amounts of data and algorithms.

[0752] "Medical information" refers to all medical information, such as a patient's health condition, diagnosis results, and treatment plan.

[0753] A "translation result" is the result of a generative model converting input text or speech into another language.

[0754] "Display in real time" means that translated text or audio is displayed immediately.

[0755] "Healthcare professional" refers to a professional person engaged in healthcare, such as a doctor, nurse, or pharmacist.

[0756] "Protection" refers to measures taken to prevent unauthorized access or leakage of data.

[0757] A "culturally sensitive algorithm" is an algorithm that takes cultural background into account when producing translations and adjusts the content to be appropriate.

[0758] "Food Menu" means a list of foods and beverages that a Customer can order.

[0759] "Restaurant staff" refers to people who work in a restaurant and take orders and serve food.

[0760] This invention is a system that enables accurate communication between patients and food delivery service users, even in different language environments. The system receives the user's language settings, translates the information based on a generative model, and displays the translation results in real time.

[0761] Hardware and Software Configuration

[0762] The following hardware and software are used to implement this system:

[0763] Hardware: Smartphones, servers

[0764] Software: Firebase, Google Translate API, SpeechRecognition library

[0765] Details of data processing and data calculation

[0766] 1. A user uses a smartphone to input voice or text, such as when the user complains of a symptom, "I have been experiencing severe headaches," or when the user orders a dish, "I would like to order a spicy ramen."

[0767] 2. The device converts the voice data into text data using the SpeechRecognition library.

[0768] 3. The server receives the data sent from the device and translates it based on the generative model. The translation process uses the Google Translate API.

[0769] 4. The translation results are displayed in real time on the user's or staff's device. The displayed information is encrypted using a secure communication protocol to protect the data.

[0770] 5. The server applies a culturally sensitive algorithm and makes any necessary adjustments to the translation result, thereby providing a translation that is appropriate for the user's cultural background.

[0771] Specific example explanation

[0772] Scenario: English-speaking foreign patient

[0773] 1. The user (foreign patient) uses the smartphone app to set their language setting to "English."

[0774] 2. The device sends the registration information to the server, which loads the English-Japanese medical terminology dictionary.

[0775] 3. During a consultation, the user speaks, "I have been experiencing severe headaches."

[0776] 4. The device converts the voice data into text and sends it to the server.

[0777] 5. The server uses the generative model to translate the sentence into Japanese: "I've been experiencing severe headaches since last week."

[0778] 6. The device displays the translation results on the medical professional's screen, and the medical professional asks any additional questions.

[0779] Scenario: An English-speaking customer uses a Japanese food delivery service.

[0780] 1. The user accesses the app and sets the language setting to "English."

[0781] 2. The device sends the information to the server, which loads the English-Japanese menu database.

[0782] 3. The user says, "I would like to order a spicy ramen."

[0783] 4. The device converts the voice data into text and sends it to the server.

[0784] 5. The server uses the generative model to translate "I would like to order spicy ramen" into Japanese.

[0785] 6. The device displays the translation results on the restaurant staff's screen, and the staff asks questions to confirm the order.

[0786] Prompt Sentence Examples

[0787] "Translate the following text from English to Japanese: 'I would like to order a spicy ramen.'"

[0788] By using this system, patients who speak different languages ​​and food delivery service users can overcome communication barriers and receive services smoothly.

[0789] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0790] Step 1:

[0791] A user accesses the app using their smartphone and selects their language preference. As input, the user enters their language preference and sends it to the server. As output, the server receives the information and loads the appropriate terminology database.

[0792] Step 2:

[0793] The device receives voice data input from the user and converts it into text data. This process uses the SpeechRecognition library. The input is voice data, and the output is the converted text data.

[0794] Step 3:

[0795] The terminal sends the converted text data to the server. There is text data as input and data sent to the server as output.

[0796] Step 4:

[0797] The server translates the received text data using a generative model. This translation process uses the Google Translate API. The input is text data, and the output is the translated text.

[0798] Step 5:

[0799] The server sends the translated text to the user's and staff's terminals in real time. The input is the translation result, and the output is the translation result displayed on the terminal.

[0800] Step 6:

[0801] The user (medical professional or staff) can check the displayed translation result and make corrections or ask additional questions if necessary. The input is the translation result, and the output is the corrected information or additional questions.

[0802] Step 7:

[0803] The server encrypts medical information and food menu data and uses a secure communication protocol to protect the data. User and staff input data is used as input, and secure data communication is used as output.

[0804] Step 8:

[0805] The server applies a culturally sensitive algorithm to the generative model and adjusts the translation results as needed. The input is the translation results and cultural data, and the output is a culturally appropriate translation result.

[0806] Step 9:

[0807] The user (patient or consumer) can use their smartphone to check the translation results and facilitate the service (medical consultation or food order).The input is the adjusted translation result, and the output is the user's understanding and next action.

[0808] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0809] This system is designed to facilitate effective communication between patients and medical professionals, and is particularly designed to enable patients who speak different languages ​​and the elderly to smoothly communicate and understand information in medical settings. Furthermore, by incorporating an emotion engine that recognizes the user's emotions, this system allows the translation results to reflect the user's emotional state, enabling more appropriate and sensitive communication.

[0810] The program of this system mainly consists of the following steps:

[0811] User Registration and Authentication

[0812] 1. User: Accesses the system and enters name, contact information, password, etc. on the account registration screen.

[0813] 2. Terminal: Sends the entered registration information to the server.

[0814] 3. Server: Stores the submitted information in a database and generates authentication information that the user will use the next time they log in.

[0815] Verifying your language preferences and needs

[0816] 1. User: After logging in, select the language you want to use on the language setting screen. For example, select "English."

[0817] 2. Terminal: Sends the selected language setting information to the server.

[0818] 3. Server: Based on the received language settings, loads the appropriate medical dictionary and medical database.

[0819] Loading the medical terminology database

[0820] 1. Server: Loads appropriate medical dictionaries and medical databases and integrates them into the generative model, enabling professional medical information translation.

[0821] Real-time conversation translation

[0822] 1. User (Patient): During the consultation, the patient enters symptoms and questions by voice or text (e.g., "I have been experiencing severe headaches for the past week").

[0823] 2. Terminal: Sends input voice and text data to the server.

[0824] 3. Server: The generative model translates the received data into the specified language (e.g., Japanese) (e.g., "I have been experiencing severe headaches since last week").

[0825] 4. Terminal: The translation results are displayed in real time on the screens of the user (medical professional) and the patient.

[0826] Check and complete translation results

[0827] 1. User (medical professional): Checks the displayed translation and makes corrections or completions as necessary, for example, checking for additional symptoms.

[0828] 2. Terminal: Send the corrected information back to the server.

[0829] 3. Server: The generative model re-translates the corrected data and generates the final translation result.

[0830] Information Protection and Security

[0831] 1. Server: All patient information and medical data is encrypted and protected using secure communication protocols.

[0832] 2. Device: Configured to ensure secure data communications.

[0833] Cultural sensitivity

[0834] 1. Server: Implements the algorithm that applies training data from cultural experts to the generative model and produces culturally sensitive translations.

[0835] 2. Generative model: Reflects cultural considerations when they are necessary for the translation results.

[0836] Emotion recognition

[0837] 1. Users: When describing symptoms or questions by voice or text, they may also express emotions (e.g., "I'm very worried about my severe headaches").

[0838] 2. Terminal: Sends emotion data along with voice and text data to the server.

[0839] 3. Server: The emotion engine recognizes the user's emotional state from speech and text and applies the analysis results to the generative model.

[0840] 4. Generative model: Appropriately reflects emotional information in the translation results, producing translations that convey more emotional nuance.

[0841] Specific examples

[0842] Scenario: An English-speaking foreign patient visits a Japanese hospital

[0843] 1. The user (patient) accesses the system and registers as a user.

[0844] 2. The device sends the registration information to the server, which stores the information in a database.

[0845] 3. The user (patient) selects "English" in the language settings, and the device sends that information to the server.

[0846] 4. The server loads the English-Japanese medical terminology dictionary and integrates it into the generative model.

[0847] 5. During a consultation, the user (patient) speaks, "I have been experiencing severe headaches for the past week. I'm very worried about it."

[0848] 6. The device sends the emotion data along with the voice data to the server, and the emotion engine analyzes the emotion "worried."

[0849] 7. Based on the data, the server translates the sentence into Japanese as "I have been experiencing severe headaches since last week and I am very worried about it."

[0850] 8. The terminal displays the translation results on the user's (medical professional's) screen.

[0851] 9. The user (medical professional) reviews the translation results and asks any additional questions or clarifications.

[0852] 10. The server encrypts all data and uses secure communication protocols.

[0853] 11. The generative model appropriately reflects cultural sensitivity where necessary.

[0854] This system will enable elderly and foreign patients to receive medical care smoothly and ensure the accuracy and reliability of medical information. In addition, by combining it with an emotion engine, more detailed communication will be possible.

[0855] The processing flow will be explained below.

[0856] Step 1:

[0857] User: Accesses the system and enters name, contact information, password, etc. on the account registration screen.

[0858] Step 2:

[0859] Terminal: Sends the entered registration information to the server.

[0860] Step 3:

[0861] Server: Stores the submitted information in a database and generates authentication information that the user will use the next time they log in.

[0862] Step 4:

[0863] User: After logging in, select the language to use on the language setting screen. For example, select "English."

[0864] Step 5:

[0865] Terminal: Sends the selected language setting information to the server.

[0866] Step 6:

[0867] Server: Loads the appropriate medical dictionary and medical database based on the selected language settings.

[0868] Step 7:

[0869] Server: Prepares for translation by integrating the loaded medical dictionary and medical database into the generative model.

[0870] Step 8:

[0871] User (Patient): During the consultation, the patient will voice or text their symptoms and questions (e.g., "I have been experiencing severe headaches for the past week. I'm very worried about it").

[0872] Step 9:

[0873] Terminal: Sends input voice and text data to the server.

[0874] Step 10:

[0875] Server: The emotion engine recognizes the emotion "worried" from speech and text, and applies the analysis results to the generative model.

[0876] Step 11:

[0877] Server: The generative model translates the received data into the specified language (e.g., Japanese) (e.g., "I've been experiencing severe headaches since last week. I'm very worried about it").

[0878] Step 12:

[0879] Terminal: Displays the translation results in real time on the user's (medical professional's) screen.

[0880] Step 13:

[0881] User (medical professional): Checks the displayed translation and makes corrections or completions as necessary, for example, checking for additional symptoms.

[0882] Step 14:

[0883] Terminal: Send the corrected information back to the server.

[0884] Step 15:

[0885] Server: The generative model re-translates the corrected data and generates the final translation result.

[0886] Step 16:

[0887] Terminal: displays the final translation results on the screen of the user (patient) and medical professional.

[0888] Step 17:

[0889] Server: Protects patient personal information and medical data by encrypting it and applying secure communication protocols.

[0890] Step 18:

[0891] Server: Implements the algorithm that applies training data provided by cultural experts to the generative model and performs translation with cultural sensitivity.

[0892] Step 19:

[0893] Generative model: Properly reflecting the content of translations that require cultural sensitivity.

[0894] In this way, the system of the present invention creates an environment where elderly and foreign patients can receive medical care smoothly, ensuring the accuracy and reliability of medical information. In addition, by combining it with an emotion engine, more detailed communication is possible.

[0895] Example 2

[0896] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0897] In today's medical environment, patients who speak different languages ​​and the elderly face difficulties in communicating appropriately in medical settings. Furthermore, simple translation without considering the patient's emotional state makes it difficult for medical professionals to understand the patient's true needs and feelings. Furthermore, confidentiality of medical information and cultural sensitivity are also important, but there is still a lack of systems that can adequately address these issues.

[0898] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0899] In this invention, the server includes means for receiving the patient's language setting, means for recognizing the patient's emotional state, means for translating medical information based on a generative model, means for displaying the translation results in real time, means for a medical professional to review and correct the translation results, means for protecting the medical information, and means for applying a culturally sensitive algorithm, thereby enabling smooth communication between patients and elderly people who speak different languages ​​in medical settings and realizing sensitive and accurate medical information transmission that takes into account the patient's emotional state and cultural background.

[0900] The "means for receiving language settings" is a function that allows the user to specify the language they want to use and transmit that information to the server.

[0901] The "means for recognizing emotional states" is a function that analyzes and identifies emotions contained in the user's statements and inputs.

[0902] "Means for translating medical information based on a generative model" refers to a function that uses a generative AI model to translate user utterances or input into a specified language.

[0903] "Means for displaying translation results in real time" refers to a function that immediately outputs the translated content to a display device.

[0904] "Means for medical professionals to check and correct translation results" is a function that allows medical professionals to check the displayed translation results and correct them if necessary.

[0905] "Measures for protecting medical information" refers to the ability to encrypt patient information and medical data and protect the confidentiality of data using secure communication protocols.

[0906] "Means for applying a culturally sensitive algorithm" refers to a function that applies an algorithm that takes cultural background into account and makes appropriate adjustments to the translation results.

[0907] The present invention provides a system that enables patients and elderly people who speak different languages ​​to smoothly communicate and understand information in medical settings. The system includes a means for receiving a patient's language preference, a means for recognizing an emotional state, a means for translating medical information based on a generative model, a means for displaying the translation results in real time, a means for a medical professional to review and correct the translation results, a means for protecting medical information, and a means for applying a culturally sensitive algorithm.

[0908] User Registration and Authentication

[0909] 1. The user accesses the system and enters their name, contact information, and password on the account registration screen. The device sends the entered registration information to the server. The server stores the received information in a database and generates a user ID and authentication token. This authentication token is used the next time the user logs in.

[0910] Verifying your language preferences and needs

[0911] 1. After logging in, the user selects the language they want to use on the language setting screen. For example, they select "English." The device sends this language setting information to the server. Based on the selected language, the server loads the appropriate medical terminology dictionary and medical database and integrates them into the generative model.

[0912] Loading the medical terminology database

[0913] 1. The server integrates medical terminology dictionaries and medical databases into the generative model to enable the translation of specialized medical information. This process ensures accurate translation of specialized terminology and medical-related information.

[0914] Real-time conversation translation

[0915] 1. During a consultation, the user (patient) inputs their symptoms or questions by voice or text. For example, they might input "I have been experiencing severe headaches for the past week." The device sends the input voice or text data to the server. The server uses a generative model to translate the received data into the specified language (for example, Japanese). The translation result is "I have been experiencing severe headaches since last week." The device displays the translation result in real time on the screens of the user (medical professional) and the patient.

[0916] Check and complete translation results

[0917] 1. The user (medical professional) checks the displayed translation results and makes corrections or additions as necessary. The device then sends the corrected information back to the server, where the generative model re-translates the corrected data to generate the final translation results.

[0918] Information Protection and Security

[0919] 1. The server encrypts all patient information and medical data and protects the data using secure communication protocols. The device is configured to ensure secure data communication.

[0920] Cultural sensitivity

[0921] 1. The server applies training data provided by cultural experts to a generative model, implementing an algorithm that achieves culturally sensitive translation. The generative model then reflects any cultural considerations that may be required in the translation results.

[0922] Emotion recognition

[0923] 1. When a user inputs symptoms or questions using voice or text, they also express their emotions. For example, they might input "I'm very worried about my severe headaches." The device sends the emotion data along with the voice and text data to the server. The server's emotion engine analyzes the user's emotional state from the voice or text and applies the analysis results to a generative model. The generative model then incorporates the emotion information into the translation results, resulting in a translation that conveys more emotional nuance.

[0924] Specific examples

[0925] Scenario: An English-speaking foreign patient visits a Japanese hospital

[0926] 1. The user (patient) accesses the system, enters their name, contact information, and password on the registration screen, enters the required information in the form, and clicks the submit button.

[0927] 2. The device sends the registration information to the server, after which a registration success message is displayed.

[0928] 3. The server saves the information to a database and generates an authentication token. The user ID and token are registered in the database.

[0929] 4. The user (patient) logs in and selects "English" in the language settings. Select English from the drop-down menu and click the Save button.

[0930] 5. The device sends the language setting information to the server.

[0931] 6. The server loads the English-Japanese medical terminology dictionary and integrates it into the generative model.

[0932] 7. During a consultation, the user (patient) dictates, "I have been experiencing severe headaches for the past week. I'm very worried about it." This can be done using a dedicated microphone or a text input field.

[0933] 8. The device sends the voice data and emotion data to the server, and the emotion engine analyzes the emotion "worried."

[0934] 9. Based on the data, the server translates the sentence into Japanese: "I have been experiencing severe headaches since last week and I am very worried about it."

[0935] 10. The device displays the translation results on the screen of the user (medical professional) and patient. The translation results are displayed in real time on the screen of a smartphone or tablet.

[0936] 11. The user (medical professional) checks the translation results and asks any additional questions or clarifications. Checks the content displayed on the tablet and enters additional instructions if necessary.

[0937] 12. The server encrypts all data and uses secure communication protocols.

[0938] 13. The generative model appropriately reflects cultural sensitivity where necessary.

[0939] Specific prompt examples

[0940] "In a scenario where an English-speaking foreign patient is seen at a hospital in Japan, and the patient says, 'I have been experiencing severe headaches for the past week. I'm very worried about it,' how would this system work?"

[0941] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0942] Step 1:

[0943] A user accesses the system and enters their name, contact information, and password on the account registration screen. The device sends the entered registration information to the server. The server stores the received information in a database and generates a user ID and authentication token.

[0944] Input: Name, Contact, Password

[0945] Output: User ID, authentication token

[0946] Specific operation: The user enters information into the form and clicks the submit button. The device sends the information to the server, which stores it in the database.

[0947] Step 2:

[0948] The user logs in and selects the language to use on the language setting screen. The terminal sends the language setting information to the server. The server loads the medical terminology dictionary and medical database based on the selected language.

[0949] Input: Login information, language setting information

[0950] Output: Medical terminology dictionary, medical database

[0951] Specific operation: A user logs in, selects the language they want to use from a drop-down menu, and clicks the Save button. The device sends the language setting information to the server, which loads the appropriate database.

[0952] Step 3:

[0953] The server integrates medical terminology dictionaries and medical databases into the generative model, preparing it for the translation of specialized medical information.

[0954] Input: Medical terminology dictionary, medical database

[0955] Output: A unified generative model

[0956] Specific operation: The server loads the medical terminology dictionary and medical database and performs the process of integrating them into the generative model.

[0957] Step 4:

[0958] During a consultation, the user (patient) inputs symptoms and questions by voice or text. The device sends the input voice or text data to the server, where a generative model translates the data and converts it into the specified language.

[0959] Input: Audio data, text data

[0960] Output: Translation data

[0961] Specific operation: The user speaks into the microphone or types into the text field. The device sends the data to the server, which then translates it using the generative model.

[0962] Step 5:

[0963] The terminal displays the translation results in real time on the screen of the user (medical professional) and the patient.

[0964] Input: Translation data

[0965] Output: Translation results displayed on the screen

[0966] Specific operation: The translation results are displayed in real time on the screen of your smartphone or tablet.

[0967] Step 6:

[0968] The user (medical professional) checks the displayed translation results and makes corrections or additions as necessary. The device then sends the corrected information back to the server, where the generative model re-translates the corrected data to generate the final translation results.

[0969] Input: Corrected translation data

[0970] Output: Final translation data

[0971] How it works: A medical professional reviews the translation results, enters corrections on the tablet, and the device sends the data back to the server, which then uses the generative model to re-translate the corrected data.

[0972] Step 7:

[0973] The server encrypts all patient information and medical data and protects the data using secure communication protocols.

[0974] Input: Patient information, medical data

[0975] Output: Encrypted data, secure communication

[0976] What happens: The server encrypts the data using an encryption algorithm and establishes a secure communication protocol.

[0977] Step 8:

[0978] The server implements an algorithm that applies training data provided by cultural experts to a generative model to achieve culturally sensitive translation.

[0979] Input: Training data

[0980] Output: Culturally sensitive algorithm

[0981] Specific operation: The server applies the training data to the generative model and updates the algorithm.

[0982] Step 9:

[0983] When users input symptoms or questions by voice or text, they also express their emotions. The device sends the emotion data along with the voice and text data to the server. The server's emotion engine analyzes the user's emotional state from the voice or text and applies the analysis results to a generative model. The generative model reflects the emotion information in the translation results, resulting in a translation that conveys more emotional nuance.

[0984] Input: Voice data, text data, emotion data

[0985] Output: Translation data reflecting emotional information

[0986] How it works: The user inputs their emotions through voice or text. The device sends the data to the server, which then uses an emotion engine to recognize the emotional state and apply it to the generative model.

[0987] (Application example 2)

[0988] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0989] The problem that this invention aims to solve is to support patients and elderly people who speak different languages ​​in smoothly communicating and understanding information in medical settings, and to realize more appropriate communication through emotion recognition. Another problem is to achieve both data protection and cultural considerations. Furthermore, it is expected that the same effect will be achieved in virtual stores as an application.

[0990] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0991] In this invention, the server includes means for receiving a patient's language setting, means for translating medical information based on a generative model, means for displaying the translation result in real time, means for a medical professional to check and correct the translation result, means for protecting the medical information, means for applying a culturally sensitive algorithm, and means for recognizing emotional states using an emotion engine and reflecting them in the translation result, thereby enabling smooth and appropriate communication between patients, elderly people, and even virtual store customers who speak different languages.

[0992] The "means for receiving patient language setting" is an interface for the patient to select or set the language they use and send that information to the system.

[0993] "Means for translating medical information based on a generative model" means a function that uses a generative AI model to translate specialized medical information into different languages ​​based on received language preferences.

[0994] "Means for displaying translation results in real time" refers to a display or user interface that instantly displays translated medical information so that patients and medical professionals can quickly review it.

[0995] "Means for medical professionals to review and correct translation results" refers to a function that allows medical professionals to review the translation results displayed by the system and correct or supplement them as necessary.

[0996] "Measures to protect medical information" refers to the use of encryption and secure communication protocols to protect patient privacy and safely handle medical information.

[0997] "Means for applying culturally sensitive algorithms" are algorithms that use training data that takes into account cross-cultural sensitivities and that reflect cultural context in translated medical information.

[0998] "Means of recognizing emotional states using an emotion engine and reflecting them in the translation results" is a function that analyzes the emotions contained in the patient's voice or text and appropriately reflects that emotional state in the translation results.

[0999] The present invention is implemented as a system for facilitating communication between patients and medical professionals who speak different languages, and can also support communication between customers and staff in virtual stores.

[1000] The system mainly consists of a server, a terminal, and a user. In this embodiment, the system is implemented using terminals such as smartphones, smart glasses, and head-mounted displays, and uses generative AI models and related algorithms on the server.

[1001] Overall system flow

[1002] 1. User Registration and Authentication

[1003] The user accesses the system and registers an account by entering their name, contact information, password, etc.

[1004] The device sends the entered information to the server, which stores it in a database and generates authentication information to use for subsequent logins.

[1005] 2. Identifying language preferences and needs

[1006] After logging in, the user sets the language to be used, for example, English or Japanese.

[1007] The terminal transmits the selected language information to the server, which then loads the corresponding medical terminology dictionary and product database.

[1008] 3. Real-time translation of conversations

[1009] Users (patients or customers) input questions or symptoms by voice or text. For example, a voice input might say, "I have been experiencing severe headaches for the past week."

[1010] The device sends the input data to the server, which then translates it using a generative AI model (using TensorFlow), and displays the translation results on the device in real time.

[1011] In addition, an emotion engine (using the Azure Emotion API) analyzes emotions and reflects them in the translation results.

[1012] 4. Check and complete the translation result

[1013] The user (medical professional or staff) reviews the translation and corrects it if necessary.

[1014] The device then sends the corrected information back to the server, and the final translation result is generated.

[1015] 5. Data Protection and Security

[1016] The server encrypts all data and protects it with secure communication protocols (SSL / TLS).

[1017] This allows for smooth and appropriate communication with patients and customers who speak different languages, and the system is also culturally sensitive, supporting more natural communication.

[1018] Specific examples

[1019] Use in medical settings

[1020] Patient (English speaker): "I have been experiencing severe headaches for the past week. I'm very worried about it."

[1021] The device sends voice and emotion data to the server, which translates it into "I've been experiencing severe headaches since last week and I'm very worried about it," and displays the results reflecting the emotion.

[1022] Use in virtual stores

[1023] Customer (English speaker): "What ingredients are in this skincare product? I have sensitive skin and I'm really worried about potential reactions."

[1024] The device sends voice and emotion data to the server, which translates it into "What ingredients are in this skin care product? I have sensitive skin, so I'm worried about a reaction," and displays the results reflecting the emotion.

[1025] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1026] Step 1:

[1027] A user accesses the system and enters their name, contact information, password, etc. on the account registration screen. The device sends this information to the server, which then stores it in a database. Authentication information to be used for subsequent logins is also generated.

[1028] Input: Name, Contact, Password

[1029] Data processing: Save the input information in the database and generate authentication information

[1030] Output: Credentials

[1031] Step 2:

[1032] After logging in, the user selects the language they want to use. The terminal sends the selected language information to the server, which then loads the corresponding medical terminology dictionary and product database based on that information, thereby preparing the necessary translation resources.

[1033] Input: Select your language

[1034] Data processing: Load medical dictionary and product database based on language settings

[1035] Output: Available translation resources

[1036] Step 3:

[1037] The user (patient or customer) inputs a question or symptom via voice or text. For example, a voice input might say, "I have been experiencing severe headaches for the past week." The device sends this to the server, which then uses a generative AI model to translate the input data.

[1038] Input: Voice or text input

[1039] Data transformation: Translating input data using generative AI models

[1040] Output: Translation result

[1041] Step 4:

[1042] The device receives the voice and text data sent to the server and uses an emotion engine to recognize the emotional state. Using this emotion data, the server reflects the emotional information in the translation results.

[1043] Input: Voice and text data

[1044] Data processing: Emotion recognition by emotion engine and reflection in translation results

[1045] Output: Translation results that reflect emotional information

[1046] Step 5:

[1047] The translation results are displayed on the device in real time. Users (medical professionals and staff) can review the translation results and make corrections as needed. The corrected information is then sent back to the server, where the final translation results are generated.

[1048] Input: Translation results that reflect emotional information

[1049] Data processing: User checks and corrects, and final translation results are generated.

[1050] Output: The final translation result

[1051] Step 6:

[1052] All data is encrypted and protected using secure communication protocols (SSL / TLS), ensuring patient and customer information is kept secure.

[1053] Input: Translation results and user information

[1054] Data processing: Encrypting data and using secure communication protocols

[1055] Output: Secure data

[1056] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1057] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1058] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1059] [Third embodiment]

[1060] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1061] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1062] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1063] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1064] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1065] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1066] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1067] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1068] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1069] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1070] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1071] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1072] The present invention is a system for effective communication between patients and medical professionals, and is designed to enable patients who speak different languages ​​and the elderly to smoothly communicate and understand information in medical settings.

[1073] The program of this system mainly consists of the following steps:

[1074] User Registration and Authentication

[1075] 1. User: Accesses the system and enters the required information (name, contact information, password, etc.) on the account registration screen.

[1076] 2. Terminal: Sends the entered user information to the server.

[1077] 3. Server: Stores user information in a database and generates and manages authentication information, which is used for subsequent logins.

[1078] Verifying your language preferences and needs

[1079] 1. User: After logging in, select the language you want to use on the language settings screen. Also, specify whether you need medical translation.

[1080] 2. Terminal: Sends the selected language setting information to the server.

[1081] 3. Server: Based on the received language settings, loads the appropriate medical dictionary and medical database.

[1082] Loading the medical terminology database

[1083] 1. Server: Loads appropriate medical dictionaries and medical databases and integrates them into the generative model, enabling professional medical information translation.

[1084] Real-time conversation translation

[1085] 1. User (Patient): During a consultation, the patient enters symptoms or questions via voice or text (e.g., "I've had a bad headache lately").

[1086] 2. Terminal: Sends input voice and text data to the server.

[1087] 3. Server: The generative model processes the received data and translates it in real time into the selected language (e.g., "I have been experiencing severe headaches recently").

[1088] 4. Terminal: Displays the translation results on the screen of the user (medical professional) and patient.

[1089] Check and complete translation results

[1090] 1. User (medical professional): Check the displayed translation results and make corrections or additions as necessary. This allows for correction of mistranslations and missing information.

[1091] 2. Terminal: Send the corrected information back to the server.

[1092] 3. Server: The generative model retranslates the corrected information and provides it to patients and specialists.

[1093] Information Protection and Security

[1094] 1. Server: All patient information and medical data is encrypted and protected using secure communication protocols.

[1095] 2. Device: Configured to ensure secure data communications.

[1096] Cultural sensitivity

[1097] 1. Server: Implements the algorithm that applies training data from cultural experts to the generative model and produces culturally sensitive translations.

[1098] 2. Generative model: Reflects cultural considerations when they are necessary for the translation results.

[1099] Specific examples

[1100] Scenario: An English-speaking foreign patient visits a Japanese hospital

[1101] 1. The user (patient) accesses the system and registers as a user.

[1102] 2. The device sends the registration information to the server, which stores the information in a database.

[1103] 3. The user (patient) selects "English" in the language settings, and the device sends that information to the server.

[1104] 4. The server loads the English-Japanese medical terminology dictionary and integrates it into the generative model.

[1105] 5. During a consultation, the user (patient) speaks, "I have been experiencing severe headaches for the past week."

[1106] 6. The device sends the voice data to the server, and the generative model translates it into Japanese as "I've been experiencing a severe headache since last week."

[1107] 7. The terminal displays the translation results on the user's (medical professional's) screen.

[1108] 8. The user (medical professional) reviews the translation and asks any additional questions.

[1109] 9. The server encrypts all data and uses secure communication protocols.

[1110] 10. The generative model appropriately reflects cultural sensitivity where necessary.

[1111] This system will enable elderly and foreign patients to receive medical care smoothly and ensure the accuracy and reliability of medical information.

[1112] The processing flow will be explained below.

[1113] Step 1:

[1114] User: Accesses the system and enters name, contact information, password, etc. on the account registration screen.

[1115] Step 2:

[1116] Terminal: Sends the entered registration information to the server.

[1117] Step 3:

[1118] Server: Stores the submitted information in a database and generates authentication information that the user will use the next time they log in.

[1119] Step 4:

[1120] User: After logging in, select the language to use on the language setting screen. For example, select "English."

[1121] Step 5:

[1122] Terminal: Sends the selected language setting information to the server.

[1123] Step 6:

[1124] Server: Loads the appropriate medical dictionary and medical database based on the selected language settings.

[1125] Step 7:

[1126] Server: Prepares for translation by integrating the loaded medical dictionary and medical database into the generative model.

[1127] Step 8:

[1128] User (Patient): During the consultation, the patient will voice or text in their symptoms and questions (e.g., "I have been experiencing severe headaches for the past week").

[1129] Step 9:

[1130] Terminal: Sends input voice and text data to the server.

[1131] Step 10:

[1132] Server: The generative model translates the received data into the specified language (e.g., Japanese) (e.g., "I have been experiencing severe headaches since last week").

[1133] Step 11:

[1134] Terminal: Displays the translation results in real time on the user's (medical professional's) screen.

[1135] Step 12:

[1136] User (medical professional): Checks the displayed translation and makes corrections or completions as necessary, for example, checking for additional symptoms.

[1137] Step 13:

[1138] Terminal: Send the corrected information back to the server.

[1139] Step 14:

[1140] Server: The generative model re-translates the corrected data and generates the final translation result.

[1141] Step 15:

[1142] Terminal: displays the final translation results on the screen of the user (patient) and medical professional.

[1143] Step 16:

[1144] Server: Protects patient personal information and medical data by encrypting it and applying secure communication protocols.

[1145] Step 17:

[1146] Server: Implements the algorithm that applies training data provided by cultural experts to the generative model and performs translation with cultural sensitivity.

[1147] Step 18:

[1148] Generative model: Properly reflecting the content of translations that require cultural sensitivity.

[1149] Through the above steps, the system of the present invention creates an environment in which elderly and foreign patients can receive medical care smoothly, and ensures the accuracy and reliability of medical information.

[1150] Example 1

[1151] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1152] Patients and the elderly who speak different languages ​​often have difficulty communicating and understanding information smoothly in medical settings. In particular, inaccurate medical terminology and explanations of symptoms can hinder diagnosis and treatment. Furthermore, translation errors and a lack of cultural sensitivity can lead to misunderstandings between patients and medical professionals.

[1153] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1154] In this invention, the server includes means for receiving a patient's language setting, means for translating medical information based on a generative AI model, means for displaying the translation results in real time, means for medical professionals to review and correct the translation results, means for protecting the medical information, means for applying a culturally sensitive algorithm, means for converting voice data to text, means for loading a medical terminology dictionary and a medical database, and means for smooth information exchange between patients and medical professionals. This enables patients and elderly people who speak different languages ​​to exchange information accurately and quickly in medical settings, improving the accuracy and reliability of medical information transmission.

[1155] The "means for receiving patient language setting" is an interface through which the system inputs and acquires information specifying the language used by the patient.

[1156] A "means for translating medical information based on a generative AI model" is a function that uses an artificial intelligence model to translate input medical-related information into different languages.

[1157] "Means for displaying the translation results in real time" refers to a technique for instantly displaying the translated information on the user's display device.

[1158] "Means for medical professionals to review and correct translation results" refers to an interface that allows medical professionals to review the translation results provided by the system and make corrections or additions as necessary.

[1159] "Measures to protect medical information" are encryption technologies and security protocols used to protect medical information, such as patient personal information and medical records, from third parties.

[1160] "Means for applying culturally sensitive algorithms" are algorithms that allow the generative AI model to take cultural background and specific context into account when translating.

[1161] "Means for converting voice data to text" refers to speech recognition technology that analyzes voice input and provides the content to the system as text data.

[1162] The "means for loading medical terminology dictionaries and medical databases" refers to a function for loading dictionaries and databases containing medical terminology and related information into the system.

[1163] "Means for smooth information exchange between patients and medical professionals" refers to interfaces and communication technologies that allow patients and medical professionals to organize information and communicate it quickly and accurately.

[1164] The present invention relates to a system for effective communication between patients and medical professionals. It is designed to enable patients who speak different languages ​​and elderly people to smoothly communicate and understand information in medical settings. The system is realized using the following main hardware and software:

[1165] Hardware and software used

[1166] Server: A high-performance backend server for handling databases, generative AI models, and encryption technologies.

[1167] Device: Computers and smart devices (e.g., PCs, tablets, smartphones) used by patients and healthcare professionals.

[1168] Generative AI models: Use OpenAI GPT-4 or similar language models to translate medical information.

[1169] Speech recognition engine: A technology that converts speech to text, such as the Google Cloud Speech-to-Text API.

[1170] Database: A relational database such as MySQL or PostgreSQL to manage user information and a medical dictionary.

[1171] Front-end technology: Frameworks for creating user interfaces, such as React and Vue.js.

[1172] Program processing flow

[1173] The system consists of the following steps:

[1174] 1. User Registration and Authentication

[1175] 1. The user (patient) accesses the system and enters the required information such as name, contact information, password, etc. The entered information is sent to the server using a secure communication protocol (SSL / TLS).

[1176] 2. The server stores the received user information in a database and generates a unique identifier (user ID).

[1177] 3. The server generates authentication information and manages the user account. This authentication information is used for subsequent logins.

[1178] 2. Identifying language preferences and needs

[1179] 1. After logging in, the user selects the language they want to use on the language settings screen. For example, they can choose English as their primary language. They can also specify whether they need medical translation.

[1180] 2. The terminal sends the selected language setting information to the server, and the server loads the appropriate medical terminology dictionary and medical database based on that information.

[1181] 3. Loading the Medical Terminology Database

[1182] 1. The server loads an open-source medical dictionary (e.g., MedDRA or SNOMED CT) according to the user's language settings.

[1183] 2. The server integrates the loaded dictionaries and databases into a generative AI model, enabling the translation of specialized medical information.

[1184] 4. Real-time translation of conversations

[1185] 1. The user (patient) enters symptoms and questions by voice or text during a consultation. For example, the user might enter "I have been experiencing severe headaches for the past week."

[1186] 2. The device sends the voice data to the server, which uses a speech recognition engine to convert the voice into text.

[1187] 3. The server uses the generative AI model to translate the text data into the selected language (e.g., Japanese) in real time.

[1188] 4. The terminal displays the translation result on the screen of the user (medical professional) and the patient. For example, the translation may read, "I have been experiencing a severe headache since last week."

[1189] 5. Check and complete the translation result

[1190] 1. The user (medical professional) checks the displayed translation and makes corrections or additions as necessary. For example, adding an additional question such as "When did it start?"

[1191] 2. The terminal sends the corrected information back to the server, and the server performs a retranslation.

[1192] 6. Data Protection and Security

[1193] 1. The server encrypts all patient information and medical data using standard encryption techniques such as AES.

[1194] 2. The server protects data using a secure communication protocol (SSL / TLS) and achieves encrypted communication using public and private keys.

[1195] 3. The device is configured to ensure secure data communications.

[1196] 7. Cultural Sensitivity

[1197] 1. The server applies training data from cultural experts to a generative AI model, implementing an algorithm that produces translations that reflect cultural nuances.

[1198] 2. The generative model will incorporate cultural considerations where necessary, for example, translating with consideration for expressions and etiquette specific to Japanese culture.

[1199] Specific examples

[1200] Scenario: An English-speaking foreign patient visits a Japanese hospital

[1201] 1. The user (patient) accesses the system and registers as a user.

[1202] 2. The device sends the registration information to the server, which stores the information in a database.

[1203] 3. The user (patient) selects "English" in the language settings, and the device sends that information to the server.

[1204] 4. The server loads an English-Japanese medical terminology dictionary and integrates it into the generative AI model.

[1205] 5. During a consultation, the user (patient) speaks, "I have been experiencing severe headaches for the past week."

[1206] 6. The device sends the voice data to the server, and the generative AI model translates it into Japanese as, "I've been experiencing a severe headache since last week."

[1207] 7. The terminal displays the translation results on the user's (medical professional's) screen.

[1208] 8. The user (medical professional) reviews the translation and asks any additional questions.

[1209] 9. The server encrypts all data and uses secure communication protocols.

[1210] 10. The generative model appropriately reflects cultural sensitivity where necessary.

[1211] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1212] Step 1:

[1213] A user accesses the system and enters the necessary information on the account registration screen, such as their name, contact information, and password. This input data is collected as the user's personal information.

[1214] Output: The entered user information is formatted as data.

[1215] Step 2:

[1216] The terminal sends the entered user information to the server, using a secure communication protocol such as SSL / TLS to ensure that the user can send data safely.

[1217] Output: The formatted user information is sent to the server as encrypted data.

[1218] Step 3:

[1219] The server stores the received user information in a database. A unique identifier (user ID) is generated for each user and authentication information is managed. This authentication information is used for subsequent logins.

[1220] Output: User information stored in the database and the generated user ID.

[1221] Step 4:

[1222] After logging in, the user selects the language to use on the language setting screen. They also specify whether they need medical translation. For example, they select English as their primary language and specify that they need translation.

[1223] Output: Selected language settings and translation needs.

[1224] Step 5:

[1225] The device sends the selected language preference information to the server, packaged in a format such as JSON and transmitted over a secure protocol.

[1226] Output: Language preference information is sent to the server.

[1227] Step 6:

[1228] The server loads the appropriate medical dictionary and medical database based on the received language setting information, for example, an English-Japanese medical dictionary.

[1229] Output: Loaded medical dictionary and database.

[1230] Step 7:

[1231] The server then integrates the loaded medical terminology dictionaries and databases into the generative AI model, passing the dictionary data as input to the model to ensure that it can translate specialized medical information.

[1232] Output: A generative AI model integrated with a medical terminology dictionary.

[1233] Step 8:

[1234] During the consultation, the user (patient) can input symptoms and questions by voice or text. For example, they can input "I have been experiencing severe headaches for the past week."

[1235] Output: Audio or text data.

[1236] Step 9:

[1237] The device sends the input voice and text data to the server. In the case of voice data, it is first converted into text using a speech recognition engine, for example, using the Google Cloud Speech-to-Text API.

[1238] Output: The audio data in text format is sent to the server.

[1239] Step 10:

[1240] The server uses a generative AI model to translate the received text data into the selected language (e.g., Japanese) in real time, for example, translating "I have been experiencing severe headaches for the past week" to "I have been experiencing severe headaches since last week."

[1241] Output: Real-time translated text data.

[1242] Step 11:

[1243] The device displays the translation results on the screen of the user (medical professional) and patient, and uses a front-end framework (such as React or Vue.js).

[1244] Output: The translation results are displayed on the screens of the medical professional and the patient.

[1245] Step 12:

[1246] The user (medical professional) checks the displayed translation and makes corrections or additions as necessary, for example adding the question "When did it start?"

[1247] Output: The modified or added text data.

[1248] Step 13:

[1249] The device then sends the corrected information back to the server, again packaged in JSON format or similar and transmitted over a secure protocol.

[1250] Output: The modified text data is sent to the server.

[1251] Step 14:

[1252] The server then uses the generative AI model to perform re-translation, translating the corrected text data back into the local language and providing accurate information to the user.

[1253] Output: The corrected translation text data.

[1254] Step 15:

[1255] The server encrypts all patient information and medical data and protects the data using secure communication protocols (SSL / TLS), specifically using encryption techniques such as AES.

[1256] Output: Encrypted user data and securely stored information.

[1257] Step 16:

[1258] The generative AI model applies training data provided by cultural experts to build algorithms that translate with cultural sensitivity, allowing translations to reflect cultural nuances, such as taking into account expressions and etiquette specific to Japanese culture.

[1259] Output: Culturally sensitive translation results.

[1260] (Application example 1)

[1261] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1262] When a language barrier exists between patients and medical professionals, it can be difficult for patients who speak different languages ​​or for the elderly to smoothly communicate and understand information in medical settings. Communication regarding the language settings of food menus is also a challenge. This can hinder smooth access to medical services and food delivery services.

[1263] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1264] In this invention, the server includes means for receiving a patient's language preference, means for translating medical information based on a generative model, means for displaying the translation result in real time, means for a medical professional to review and correct the translation result, means for protecting the medical information, means for applying a culturally sensitive algorithm, means for receiving a food menu's language preference, means for translating the food information based on a generative model, and means for displaying the translation result to restaurant staff, thereby enabling accurate communication of information between patients and food delivery service users in different language environments.

[1265] A "patient" is a person receiving medical services.

[1266] "Language settings" refers to settings that specify the language used by the user.

[1267] A "generative model" is an artificial intelligence that generates and translates text and speech based on large amounts of data and algorithms.

[1268] "Medical information" refers to all medical information, such as a patient's health condition, diagnosis results, and treatment plan.

[1269] A "translation result" is the result of a generative model converting input text or speech into another language.

[1270] "Display in real time" means that translated text or audio is displayed immediately.

[1271] "Healthcare professional" refers to a professional person engaged in healthcare, such as a doctor, nurse, or pharmacist.

[1272] "Protection" refers to measures taken to prevent unauthorized access or leakage of data.

[1273] A "culturally sensitive algorithm" is an algorithm that takes cultural background into account when producing translations and adjusts the content to be appropriate.

[1274] "Food Menu" means a list of foods and beverages that a Customer can order.

[1275] "Restaurant staff" refers to people who work in a restaurant and take orders and serve food.

[1276] This invention is a system that enables accurate communication between patients and food delivery service users, even in different language environments. The system receives the user's language settings, translates the information based on a generative model, and displays the translation results in real time.

[1277] Hardware and Software Configuration

[1278] The following hardware and software are used to implement this system:

[1279] Hardware: Smartphones, servers

[1280] Software: Firebase, Google Translate API, SpeechRecognition library

[1281] Details of data processing and data calculation

[1282] 1. A user uses a smartphone to input voice or text, such as when the user complains of a symptom, "I have been experiencing severe headaches," or when the user orders a dish, "I would like to order a spicy ramen."

[1283] 2. The device converts the voice data into text data using the SpeechRecognition library.

[1284] 3. The server receives the data sent from the device and translates it based on the generative model. The translation process uses the Google Translate API.

[1285] 4. The translation results are displayed in real time on the user's or staff's device. The displayed information is encrypted using a secure communication protocol to protect the data.

[1286] 5. The server applies a culturally sensitive algorithm and makes any necessary adjustments to the translation result, thereby providing a translation that is appropriate for the user's cultural background.

[1287] Specific example explanation

[1288] Scenario: English-speaking foreign patient

[1289] 1. The user (foreign patient) uses the smartphone app to set their language setting to "English."

[1290] 2. The device sends the registration information to the server, which loads the English-Japanese medical terminology dictionary.

[1291] 3. During a consultation, the user speaks, "I have been experiencing severe headaches."

[1292] 4. The device converts the voice data into text and sends it to the server.

[1293] 5. The server uses the generative model to translate the sentence into Japanese: "I've been experiencing severe headaches since last week."

[1294] 6. The device displays the translation results on the medical professional's screen, and the medical professional asks any additional questions.

[1295] Scenario: An English-speaking customer uses a Japanese food delivery service.

[1296] 1. The user accesses the app and sets the language setting to "English."

[1297] 2. The device sends the information to the server, which loads the English-Japanese menu database.

[1298] 3. The user says, "I would like to order a spicy ramen."

[1299] 4. The device converts the voice data into text and sends it to the server.

[1300] 5. The server uses the generative model to translate "I would like to order spicy ramen" into Japanese.

[1301] 6. The device displays the translation results on the restaurant staff's screen, and the staff asks questions to confirm the order.

[1302] Prompt Sentence Examples

[1303] "Translate the following text from English to Japanese: 'I would like to order a spicy ramen.'"

[1304] By using this system, patients who speak different languages ​​and food delivery service users can overcome communication barriers and receive services smoothly.

[1305] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1306] Step 1:

[1307] A user accesses the app using their smartphone and selects their language preference. As input, the user enters their language preference and sends it to the server. As output, the server receives the information and loads the appropriate terminology database.

[1308] Step 2:

[1309] The device receives voice data input from the user and converts it into text data. This process uses the SpeechRecognition library. The input is voice data, and the output is the converted text data.

[1310] Step 3:

[1311] The terminal sends the converted text data to the server. There is text data as input and data sent to the server as output.

[1312] Step 4:

[1313] The server translates the received text data using a generative model. This translation process uses the Google Translate API. The input is text data, and the output is the translated text.

[1314] Step 5:

[1315] The server sends the translated text to the user's and staff's terminals in real time. The input is the translation result, and the output is the translation result displayed on the terminal.

[1316] Step 6:

[1317] The user (medical professional or staff) can check the displayed translation result and make corrections or ask additional questions if necessary. The input is the translation result, and the output is the corrected information or additional questions.

[1318] Step 7:

[1319] The server encrypts medical information and food menu data and uses a secure communication protocol to protect the data. User and staff input data is used as input, and secure data communication is used as output.

[1320] Step 8:

[1321] The server applies a culturally sensitive algorithm to the generative model and adjusts the translation results as needed. The input is the translation results and cultural data, and the output is a culturally appropriate translation result.

[1322] Step 9:

[1323] The user (patient or consumer) can use their smartphone to check the translation results and facilitate the service (medical consultation or food order).The input is the adjusted translation result, and the output is the user's understanding and next action.

[1324] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1325] This system is designed to facilitate effective communication between patients and medical professionals, and is particularly designed to enable patients who speak different languages ​​and the elderly to smoothly communicate and understand information in medical settings. Furthermore, by incorporating an emotion engine that recognizes the user's emotions, this system allows the translation results to reflect the user's emotional state, enabling more appropriate and sensitive communication.

[1326] The program of this system mainly consists of the following steps:

[1327] User Registration and Authentication

[1328] 1. User: Accesses the system and enters name, contact information, password, etc. on the account registration screen.

[1329] 2. Terminal: Sends the entered registration information to the server.

[1330] 3. Server: Stores the submitted information in a database and generates authentication information that the user will use the next time they log in.

[1331] Verifying your language preferences and needs

[1332] 1. User: After logging in, select the language you want to use on the language setting screen. For example, select "English."

[1333] 2. Terminal: Sends the selected language setting information to the server.

[1334] 3. Server: Based on the received language settings, loads the appropriate medical dictionary and medical database.

[1335] Loading the medical terminology database

[1336] 1. Server: Loads appropriate medical dictionaries and medical databases and integrates them into the generative model, enabling professional medical information translation.

[1337] Real-time conversation translation

[1338] 1. User (Patient): During the consultation, the patient enters symptoms and questions by voice or text (e.g., "I have been experiencing severe headaches for the past week").

[1339] 2. Terminal: Sends input voice and text data to the server.

[1340] 3. Server: The generative model translates the received data into the specified language (e.g., Japanese) (e.g., "I have been experiencing severe headaches since last week").

[1341] 4. Terminal: The translation results are displayed in real time on the screens of the user (medical professional) and the patient.

[1342] Check and complete translation results

[1343] 1. User (medical professional): Checks the displayed translation and makes corrections or completions as necessary, for example, checking for additional symptoms.

[1344] 2. Terminal: Send the corrected information back to the server.

[1345] 3. Server: The generative model re-translates the corrected data and generates the final translation result.

[1346] Information Protection and Security

[1347] 1. Server: All patient information and medical data is encrypted and protected using secure communication protocols.

[1348] 2. Device: Configured to ensure secure data communications.

[1349] Cultural sensitivity

[1350] 1. Server: Implements the algorithm that applies training data from cultural experts to the generative model and produces culturally sensitive translations.

[1351] 2. Generative model: Reflects cultural considerations when they are necessary for the translation results.

[1352] Emotion recognition

[1353] 1. Users: When describing symptoms or questions by voice or text, they may also express emotions (e.g., "I'm very worried about my severe headaches").

[1354] 2. Terminal: Sends emotion data along with voice and text data to the server.

[1355] 3. Server: The emotion engine recognizes the user's emotional state from speech and text and applies the analysis results to the generative model.

[1356] 4. Generative model: Appropriately reflects emotional information in the translation results, producing translations that convey more emotional nuance.

[1357] Specific examples

[1358] Scenario: An English-speaking foreign patient visits a Japanese hospital

[1359] 1. The user (patient) accesses the system and registers as a user.

[1360] 2. The device sends the registration information to the server, which stores the information in a database.

[1361] 3. The user (patient) selects "English" in the language settings, and the device sends that information to the server.

[1362] 4. The server loads the English-Japanese medical terminology dictionary and integrates it into the generative model.

[1363] 5. During a consultation, the user (patient) speaks, "I have been experiencing severe headaches for the past week. I'm very worried about it."

[1364] 6. The device sends the emotion data along with the voice data to the server, and the emotion engine analyzes the emotion "worried."

[1365] 7. Based on the data, the server translates the sentence into Japanese as "I have been experiencing severe headaches since last week and I am very worried about it."

[1366] 8. The terminal displays the translation results on the user's (medical professional's) screen.

[1367] 9. The user (medical professional) reviews the translation results and asks any additional questions or clarifications.

[1368] 10. The server encrypts all data and uses secure communication protocols.

[1369] 11. The generative model appropriately reflects cultural sensitivity where necessary.

[1370] This system will enable elderly and foreign patients to receive medical care smoothly and ensure the accuracy and reliability of medical information. In addition, by combining it with an emotion engine, more detailed communication will be possible.

[1371] The processing flow will be explained below.

[1372] Step 1:

[1373] User: Accesses the system and enters name, contact information, password, etc. on the account registration screen.

[1374] Step 2:

[1375] Terminal: Sends the entered registration information to the server.

[1376] Step 3:

[1377] Server: Stores the submitted information in a database and generates authentication information that the user will use the next time they log in.

[1378] Step 4:

[1379] User: After logging in, select the language to use on the language setting screen. For example, select "English."

[1380] Step 5:

[1381] Terminal: Sends the selected language setting information to the server.

[1382] Step 6:

[1383] Server: Loads the appropriate medical dictionary and medical database based on the selected language settings.

[1384] Step 7:

[1385] Server: Prepares for translation by integrating the loaded medical dictionary and medical database into the generative model.

[1386] Step 8:

[1387] User (Patient): During the consultation, the patient will voice or text their symptoms and questions (e.g., "I have been experiencing severe headaches for the past week. I'm very worried about it").

[1388] Step 9:

[1389] Terminal: Sends input voice and text data to the server.

[1390] Step 10:

[1391] Server: The emotion engine recognizes the emotion "worried" from speech and text, and applies the analysis results to the generative model.

[1392] Step 11:

[1393] Server: The generative model translates the received data into the specified language (e.g., Japanese) (e.g., "I've been experiencing severe headaches since last week. I'm very worried about it").

[1394] Step 12:

[1395] Terminal: Displays the translation results in real time on the user's (medical professional's) screen.

[1396] Step 13:

[1397] User (medical professional): Checks the displayed translation and makes corrections or completions as necessary, for example, checking for additional symptoms.

[1398] Step 14:

[1399] Terminal: Send the corrected information back to the server.

[1400] Step 15:

[1401] Server: The generative model re-translates the corrected data and generates the final translation result.

[1402] Step 16:

[1403] Terminal: displays the final translation results on the screen of the user (patient) and medical professional.

[1404] Step 17:

[1405] Server: Protects patient personal information and medical data by encrypting it and applying secure communication protocols.

[1406] Step 18:

[1407] Server: Implements the algorithm that applies training data provided by cultural experts to the generative model and performs translation with cultural sensitivity.

[1408] Step 19:

[1409] Generative model: Properly reflecting the content of translations that require cultural sensitivity.

[1410] In this way, the system of the present invention creates an environment where elderly and foreign patients can receive medical care smoothly, ensuring the accuracy and reliability of medical information. In addition, by combining it with an emotion engine, more detailed communication is possible.

[1411] Example 2

[1412] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1413] In today's medical environment, patients who speak different languages ​​and the elderly face difficulties in communicating appropriately in medical settings. Furthermore, simple translation without considering the patient's emotional state makes it difficult for medical professionals to understand the patient's true needs and feelings. Furthermore, confidentiality of medical information and cultural sensitivity are also important, but there is still a lack of systems that can adequately address these issues.

[1414] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1415] In this invention, the server includes means for receiving the patient's language setting, means for recognizing the patient's emotional state, means for translating medical information based on a generative model, means for displaying the translation results in real time, means for a medical professional to review and correct the translation results, means for protecting the medical information, and means for applying a culturally sensitive algorithm, thereby enabling smooth communication between patients and elderly people who speak different languages ​​in medical settings and realizing sensitive and accurate medical information transmission that takes into account the patient's emotional state and cultural background.

[1416] The "means for receiving language settings" is a function that allows the user to specify the language they want to use and transmit that information to the server.

[1417] The "means for recognizing emotional states" is a function that analyzes and identifies emotions contained in the user's statements and inputs.

[1418] "Means for translating medical information based on a generative model" refers to a function that uses a generative AI model to translate user utterances or input into a specified language.

[1419] "Means for displaying translation results in real time" refers to a function that immediately outputs the translated content to a display device.

[1420] "Means for medical professionals to check and correct translation results" is a function that allows medical professionals to check the displayed translation results and correct them if necessary.

[1421] "Measures for protecting medical information" refers to the ability to encrypt patient information and medical data and protect the confidentiality of data using secure communication protocols.

[1422] "Means for applying a culturally sensitive algorithm" refers to a function that applies an algorithm that takes cultural background into account and makes appropriate adjustments to the translation results.

[1423] The present invention provides a system that enables patients and elderly people who speak different languages ​​to smoothly communicate and understand information in medical settings. The system includes a means for receiving a patient's language preference, a means for recognizing an emotional state, a means for translating medical information based on a generative model, a means for displaying the translation results in real time, a means for a medical professional to review and correct the translation results, a means for protecting medical information, and a means for applying a culturally sensitive algorithm.

[1424] User Registration and Authentication

[1425] 1. The user accesses the system and enters their name, contact information, and password on the account registration screen. The device sends the entered registration information to the server. The server stores the received information in a database and generates a user ID and authentication token. This authentication token is used the next time the user logs in.

[1426] Verifying your language preferences and needs

[1427] 1. After logging in, the user selects the language they want to use on the language setting screen. For example, they select "English." The device sends this language setting information to the server. Based on the selected language, the server loads the appropriate medical terminology dictionary and medical database and integrates them into the generative model.

[1428] Loading the medical terminology database

[1429] 1. The server integrates medical terminology dictionaries and medical databases into the generative model to enable the translation of specialized medical information. This process ensures accurate translation of specialized terminology and medical-related information.

[1430] Real-time conversation translation

[1431] 1. During a consultation, the user (patient) inputs their symptoms or questions by voice or text. For example, they might input "I have been experiencing severe headaches for the past week." The device sends the input voice or text data to the server. The server uses a generative model to translate the received data into the specified language (for example, Japanese). The translation result is "I have been experiencing severe headaches since last week." The device displays the translation result in real time on the screens of the user (medical professional) and the patient.

[1432] Check and complete translation results

[1433] 1. The user (medical professional) checks the displayed translation results and makes corrections or additions as necessary. The device then sends the corrected information back to the server, where the generative model re-translates the corrected data to generate the final translation results.

[1434] Information Protection and Security

[1435] 1. The server encrypts all patient information and medical data and protects the data using secure communication protocols. The device is configured to ensure secure data communication.

[1436] Cultural sensitivity

[1437] 1. The server applies training data provided by cultural experts to a generative model, implementing an algorithm that achieves culturally sensitive translation. The generative model then reflects any cultural considerations that may be required in the translation results.

[1438] Emotion recognition

[1439] 1. When a user inputs symptoms or questions using voice or text, they also express their emotions. For example, they might input "I'm very worried about my severe headaches." The device sends the emotion data along with the voice and text data to the server. The server's emotion engine analyzes the user's emotional state from the voice or text and applies the analysis results to a generative model. The generative model then incorporates the emotion information into the translation results, resulting in a translation that conveys more emotional nuance.

[1440] Specific examples

[1441] Scenario: An English-speaking foreign patient visits a Japanese hospital

[1442] 1. The user (patient) accesses the system, enters their name, contact information, and password on the registration screen, enters the required information in the form, and clicks the submit button.

[1443] 2. The device sends the registration information to the server, after which a registration success message is displayed.

[1444] 3. The server saves the information to a database and generates an authentication token. The user ID and token are registered in the database.

[1445] 4. The user (patient) logs in and selects "English" in the language settings. Select English from the drop-down menu and click the Save button.

[1446] 5. The device sends the language setting information to the server.

[1447] 6. The server loads the English-Japanese medical terminology dictionary and integrates it into the generative model.

[1448] 7. During a consultation, the user (patient) dictates, "I have been experiencing severe headaches for the past week. I'm very worried about it." This can be done using a dedicated microphone or a text input field.

[1449] 8. The device sends the voice data and emotion data to the server, and the emotion engine analyzes the emotion "worried."

[1450] 9. Based on the data, the server translates the sentence into Japanese: "I have been experiencing severe headaches since last week and I am very worried about it."

[1451] 10. The device displays the translation results on the screen of the user (medical professional) and patient. The translation results are displayed in real time on the screen of a smartphone or tablet.

[1452] 11. The user (medical professional) checks the translation results and asks any additional questions or clarifications. Checks the content displayed on the tablet and enters additional instructions if necessary.

[1453] 12. The server encrypts all data and uses secure communication protocols.

[1454] 13. The generative model appropriately reflects cultural sensitivity where necessary.

[1455] Specific prompt examples

[1456] "In a scenario where an English-speaking foreign patient is seen at a hospital in Japan, and the patient says, 'I have been experiencing severe headaches for the past week. I'm very worried about it,' how would this system work?"

[1457] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1458] Step 1:

[1459] A user accesses the system and enters their name, contact information, and password on the account registration screen. The device sends the entered registration information to the server. The server stores the received information in a database and generates a user ID and authentication token.

[1460] Input: Name, Contact, Password

[1461] Output: User ID, authentication token

[1462] Specific operation: The user enters information into the form and clicks the submit button. The device sends the information to the server, which stores it in the database.

[1463] Step 2:

[1464] The user logs in and selects the language to use on the language setting screen. The terminal sends the language setting information to the server. The server loads the medical terminology dictionary and medical database based on the selected language.

[1465] Input: Login information, language setting information

[1466] Output: Medical terminology dictionary, medical database

[1467] Specific operation: A user logs in, selects the language they want to use from a drop-down menu, and clicks the Save button. The device sends the language setting information to the server, which loads the appropriate database.

[1468] Step 3:

[1469] The server integrates medical terminology dictionaries and medical databases into the generative model, preparing it for the translation of specialized medical information.

[1470] Input: Medical terminology dictionary, medical database

[1471] Output: A unified generative model

[1472] Specific operation: The server loads the medical terminology dictionary and medical database and performs the process of integrating them into the generative model.

[1473] Step 4:

[1474] During a consultation, the user (patient) inputs symptoms and questions by voice or text. The device sends the input voice or text data to the server, where a generative model translates the data and converts it into the specified language.

[1475] Input: Audio data, text data

[1476] Output: Translation data

[1477] Specific operation: The user speaks into the microphone or types into the text field. The device sends the data to the server, which then translates it using the generative model.

[1478] Step 5:

[1479] The terminal displays the translation results in real time on the screen of the user (medical professional) and the patient.

[1480] Input: Translation data

[1481] Output: Translation results displayed on the screen

[1482] Specific operation: The translation results are displayed in real time on the screen of your smartphone or tablet.

[1483] Step 6:

[1484] The user (medical professional) checks the displayed translation results and makes corrections or additions as necessary. The device then sends the corrected information back to the server, where the generative model re-translates the corrected data to generate the final translation results.

[1485] Input: Corrected translation data

[1486] Output: Final translation data

[1487] How it works: A medical professional reviews the translation results, enters corrections on the tablet, and the device sends the data back to the server, which then uses the generative model to re-translate the corrected data.

[1488] Step 7:

[1489] The server encrypts all patient information and medical data and protects the data using secure communication protocols.

[1490] Input: Patient information, medical data

[1491] Output: Encrypted data, secure communication

[1492] What happens: The server encrypts the data using an encryption algorithm and establishes a secure communication protocol.

[1493] Step 8:

[1494] The server implements an algorithm that applies training data provided by cultural experts to a generative model to achieve culturally sensitive translation.

[1495] Input: Training data

[1496] Output: Culturally sensitive algorithm

[1497] Specific operation: The server applies the training data to the generative model and updates the algorithm.

[1498] Step 9:

[1499] When users input symptoms or questions by voice or text, they also express their emotions. The device sends the emotion data along with the voice and text data to the server. The server's emotion engine analyzes the user's emotional state from the voice or text and applies the analysis results to a generative model. The generative model reflects the emotion information in the translation results, resulting in a translation that conveys more emotional nuance.

[1500] Input: Voice data, text data, emotion data

[1501] Output: Translation data reflecting emotional information

[1502] How it works: The user inputs their emotions through voice or text. The device sends the data to the server, which then uses an emotion engine to recognize the emotional state and apply it to the generative model.

[1503] (Application example 2)

[1504] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1505] The problem that this invention aims to solve is to support patients and elderly people who speak different languages ​​in smoothly communicating and understanding information in medical settings, and to realize more appropriate communication through emotion recognition. Another problem is to achieve both data protection and cultural considerations. Furthermore, it is expected that the same effect will be achieved in virtual stores as an application.

[1506] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1507] In this invention, the server includes means for receiving a patient's language setting, means for translating medical information based on a generative model, means for displaying the translation result in real time, means for a medical professional to check and correct the translation result, means for protecting the medical information, means for applying a culturally sensitive algorithm, and means for recognizing emotional states using an emotion engine and reflecting them in the translation result, thereby enabling smooth and appropriate communication between patients, elderly people, and even virtual store customers who speak different languages.

[1508] The "means for receiving patient language setting" is an interface for the patient to select or set the language they use and send that information to the system.

[1509] "Means for translating medical information based on a generative model" means a function that uses a generative AI model to translate specialized medical information into different languages ​​based on received language preferences.

[1510] "Means for displaying translation results in real time" refers to a display or user interface that instantly displays translated medical information so that patients and medical professionals can quickly review it.

[1511] "Means for medical professionals to review and correct translation results" refers to a function that allows medical professionals to review the translation results displayed by the system and correct or supplement them as necessary.

[1512] "Measures to protect medical information" refers to the use of encryption and secure communication protocols to protect patient privacy and safely handle medical information.

[1513] "Means for applying culturally sensitive algorithms" are algorithms that use training data that takes into account cross-cultural sensitivities and that reflect cultural context in translated medical information.

[1514] "Means of recognizing emotional states using an emotion engine and reflecting them in the translation results" is a function that analyzes the emotions contained in the patient's voice or text and appropriately reflects that emotional state in the translation results.

[1515] The present invention is implemented as a system for facilitating communication between patients and medical professionals who speak different languages, and can also support communication between customers and staff in virtual stores.

[1516] The system mainly consists of a server, a terminal, and a user. In this embodiment, the system is implemented using terminals such as smartphones, smart glasses, and head-mounted displays, and uses generative AI models and related algorithms on the server.

[1517] Overall system flow

[1518] 1. User Registration and Authentication

[1519] The user accesses the system and registers an account by entering their name, contact information, password, etc.

[1520] The device sends the entered information to the server, which stores it in a database and generates authentication information to use for subsequent logins.

[1521] 2. Identifying language preferences and needs

[1522] After logging in, the user sets the language to be used, for example, English or Japanese.

[1523] The terminal transmits the selected language information to the server, which then loads the corresponding medical terminology dictionary and product database.

[1524] 3. Real-time translation of conversations

[1525] Users (patients or customers) input questions or symptoms by voice or text. For example, a voice input might say, "I have been experiencing severe headaches for the past week."

[1526] The device sends the input data to the server, which then translates it using a generative AI model (using TensorFlow), and displays the translation results on the device in real time.

[1527] In addition, an emotion engine (using the Azure Emotion API) analyzes emotions and reflects them in the translation results.

[1528] 4. Check and complete the translation result

[1529] The user (medical professional or staff) reviews the translation and corrects it if necessary.

[1530] The device then sends the corrected information back to the server, and the final translation result is generated.

[1531] 5. Data Protection and Security

[1532] The server encrypts all data and protects it with secure communication protocols (SSL / TLS).

[1533] This allows for smooth and appropriate communication with patients and customers who speak different languages, and the system is also culturally sensitive, supporting more natural communication.

[1534] Specific examples

[1535] Use in medical settings

[1536] Patient (English speaker): "I have been experiencing severe headaches for the past week. I'm very worried about it."

[1537] The device sends voice and emotion data to the server, which translates it into "I've been experiencing severe headaches since last week and I'm very worried about it," and displays the results reflecting the emotion.

[1538] Use in virtual stores

[1539] Customer (English speaker): "What ingredients are in this skincare product? I have sensitive skin and I'm really worried about potential reactions."

[1540] The device sends voice and emotion data to the server, which translates it into "What ingredients are in this skin care product? I have sensitive skin, so I'm worried about a reaction," and displays the results reflecting the emotion.

[1541] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1542] Step 1:

[1543] A user accesses the system and enters their name, contact information, password, etc. on the account registration screen. The device sends this information to the server, which then stores it in a database. Authentication information to be used for subsequent logins is also generated.

[1544] Input: Name, Contact, Password

[1545] Data processing: Save the input information in the database and generate authentication information

[1546] Output: Credentials

[1547] Step 2:

[1548] After logging in, the user selects the language they want to use. The terminal sends the selected language information to the server, which then loads the corresponding medical terminology dictionary and product database based on that information, thereby preparing the necessary translation resources.

[1549] Input: Select your language

[1550] Data processing: Load medical dictionary and product database based on language settings

[1551] Output: Available translation resources

[1552] Step 3:

[1553] The user (patient or customer) inputs a question or symptom via voice or text. For example, a voice input might say, "I have been experiencing severe headaches for the past week." The device sends this to the server, which then uses a generative AI model to translate the input data.

[1554] Input: Voice or text input

[1555] Data transformation: Translating input data using generative AI models

[1556] Output: Translation result

[1557] Step 4:

[1558] The device receives the voice and text data sent to the server and uses an emotion engine to recognize the emotional state. Using this emotion data, the server reflects the emotional information in the translation results.

[1559] Input: Voice and text data

[1560] Data processing: Emotion recognition by emotion engine and reflection in translation results

[1561] Output: Translation results that reflect emotional information

[1562] Step 5:

[1563] The translation results are displayed on the device in real time. Users (medical professionals and staff) can review the translation results and make corrections as needed. The corrected information is then sent back to the server, where the final translation results are generated.

[1564] Input: Translation results that reflect emotional information

[1565] Data processing: User checks and corrects, and final translation results are generated.

[1566] Output: The final translation result

[1567] Step 6:

[1568] All data is encrypted and protected using secure communication protocols (SSL / TLS), ensuring patient and customer information is kept secure.

[1569] Input: Translation results and user information

[1570] Data processing: Encrypting data and using secure communication protocols

[1571] Output: Secure data

[1572] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1573] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1574] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1575] [Fourth embodiment]

[1576] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1577] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1578] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1579] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1580] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1581] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1582] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1583] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1584] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1585] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1586] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1587] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1588] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1589] The present invention is a system for effective communication between patients and medical professionals, and is designed to enable patients who speak different languages ​​and the elderly to smoothly communicate and understand information in medical settings.

[1590] The program of this system mainly consists of the following steps:

[1591] User Registration and Authentication

[1592] 1. User: Accesses the system and enters the required information (name, contact information, password, etc.) on the account registration screen.

[1593] 2. Terminal: Sends the entered user information to the server.

[1594] 3. Server: Stores user information in a database and generates and manages authentication information, which is used for subsequent logins.

[1595] Verifying your language preferences and needs

[1596] 1. User: After logging in, select the language you want to use on the language settings screen. Also, specify whether you need medical translation.

[1597] 2. Terminal: Sends the selected language setting information to the server.

[1598] 3. Server: Based on the received language settings, loads the appropriate medical dictionary and medical database.

[1599] Loading the medical terminology database

[1600] 1. Server: Loads appropriate medical dictionaries and medical databases and integrates them into the generative model, enabling professional medical information translation.

[1601] Real-time conversation translation

[1602] 1. User (Patient): During a consultation, the patient enters symptoms or questions via voice or text (e.g., "I've had a bad headache lately").

[1603] 2. Terminal: Sends input voice and text data to the server.

[1604] 3. Server: The generative model processes the received data and translates it in real time into the selected language (e.g., "I have been experiencing severe headaches recently").

[1605] 4. Terminal: Displays the translation results on the screen of the user (medical professional) and patient.

[1606] Check and complete translation results

[1607] 1. User (medical professional): Check the displayed translation results and make corrections or additions as necessary. This allows for correction of mistranslations and missing information.

[1608] 2. Terminal: Send the corrected information back to the server.

[1609] 3. Server: The generative model retranslates the corrected information and provides it to patients and specialists.

[1610] Information Protection and Security

[1611] 1. Server: All patient information and medical data is encrypted and protected using secure communication protocols.

[1612] 2. Device: Configured to ensure secure data communications.

[1613] Cultural sensitivity

[1614] 1. Server: Implements the algorithm that applies training data from cultural experts to the generative model and produces culturally sensitive translations.

[1615] 2. Generative model: Reflects cultural considerations when they are necessary for the translation results.

[1616] Specific examples

[1617] Scenario: An English-speaking foreign patient visits a Japanese hospital

[1618] 1. The user (patient) accesses the system and registers as a user.

[1619] 2. The device sends the registration information to the server, which stores the information in a database.

[1620] 3. The user (patient) selects "English" in the language settings, and the device sends that information to the server.

[1621] 4. The server loads the English-Japanese medical terminology dictionary and integrates it into the generative model.

[1622] 5. During a consultation, the user (patient) speaks, "I have been experiencing severe headaches for the past week."

[1623] 6. The device sends the voice data to the server, and the generative model translates it into Japanese as "I've been experiencing a severe headache since last week."

[1624] 7. The terminal displays the translation results on the user's (medical professional's) screen.

[1625] 8. The user (medical professional) reviews the translation and asks any additional questions.

[1626] 9. The server encrypts all data and uses secure communication protocols.

[1627] 10. The generative model appropriately reflects cultural sensitivity where necessary.

[1628] This system will enable elderly and foreign patients to receive medical care smoothly and ensure the accuracy and reliability of medical information.

[1629] The processing flow will be explained below.

[1630] Step 1:

[1631] User: Accesses the system and enters name, contact information, password, etc. on the account registration screen.

[1632] Step 2:

[1633] Terminal: Sends the entered registration information to the server.

[1634] Step 3:

[1635] Server: Stores the submitted information in a database and generates authentication information that the user will use the next time they log in.

[1636] Step 4:

[1637] User: After logging in, select the language to use on the language setting screen. For example, select "English."

[1638] Step 5:

[1639] Terminal: Sends the selected language setting information to the server.

[1640] Step 6:

[1641] Server: Loads the appropriate medical dictionary and medical database based on the selected language settings.

[1642] Step 7:

[1643] Server: Prepares for translation by integrating the loaded medical dictionary and medical database into the generative model.

[1644] Step 8:

[1645] User (Patient): During the consultation, the patient will voice or text in their symptoms and questions (e.g., "I have been experiencing severe headaches for the past week").

[1646] Step 9:

[1647] Terminal: Sends input voice and text data to the server.

[1648] Step 10:

[1649] Server: The generative model translates the received data into the specified language (e.g., Japanese) (e.g., "I have been experiencing severe headaches since last week").

[1650] Step 11:

[1651] Terminal: Displays the translation results in real time on the user's (medical professional's) screen.

[1652] Step 12:

[1653] User (medical professional): Checks the displayed translation and makes corrections or completions as necessary, for example, checking for additional symptoms.

[1654] Step 13:

[1655] Terminal: Send the corrected information back to the server.

[1656] Step 14:

[1657] Server: The generative model re-translates the corrected data and generates the final translation result.

[1658] Step 15:

[1659] Terminal: displays the final translation results on the screen of the user (patient) and medical professional.

[1660] Step 16:

[1661] Server: Protects patient personal information and medical data by encrypting it and applying secure communication protocols.

[1662] Step 17:

[1663] Server: Implements the algorithm that applies training data provided by cultural experts to the generative model and performs translation with cultural sensitivity.

[1664] Step 18:

[1665] Generative model: Properly reflecting the content of translations that require cultural sensitivity.

[1666] Through the above steps, the system of the present invention creates an environment in which elderly and foreign patients can receive medical care smoothly, and ensures the accuracy and reliability of medical information.

[1667] Example 1

[1668] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1669] Patients and the elderly who speak different languages ​​often have difficulty communicating and understanding information smoothly in medical settings. In particular, inaccurate medical terminology and explanations of symptoms can hinder diagnosis and treatment. Furthermore, translation errors and a lack of cultural sensitivity can lead to misunderstandings between patients and medical professionals.

[1670] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1671] In this invention, the server includes means for receiving a patient's language setting, means for translating medical information based on a generative AI model, means for displaying the translation results in real time, means for medical professionals to review and correct the translation results, means for protecting the medical information, means for applying a culturally sensitive algorithm, means for converting voice data to text, means for loading a medical terminology dictionary and a medical database, and means for smooth information exchange between patients and medical professionals. This enables patients and elderly people who speak different languages ​​to exchange information accurately and quickly in medical settings, improving the accuracy and reliability of medical information transmission.

[1672] The "means for receiving patient language setting" is an interface through which the system inputs and acquires information specifying the language used by the patient.

[1673] A "means for translating medical information based on a generative AI model" is a function that uses an artificial intelligence model to translate input medical-related information into different languages.

[1674] "Means for displaying the translation results in real time" refers to a technique for instantly displaying the translated information on the user's display device.

[1675] "Means for medical professionals to review and correct translation results" refers to an interface that allows medical professionals to review the translation results provided by the system and make corrections or additions as necessary.

[1676] "Measures to protect medical information" are encryption technologies and security protocols used to protect medical information, such as patient personal information and medical records, from third parties.

[1677] "Means for applying culturally sensitive algorithms" are algorithms that allow the generative AI model to take cultural background and specific context into account when translating.

[1678] "Means for converting voice data to text" refers to speech recognition technology that analyzes voice input and provides the content to the system as text data.

[1679] The "means for loading medical terminology dictionaries and medical databases" refers to a function for loading dictionaries and databases containing medical terminology and related information into the system.

[1680] "Means for smooth information exchange between patients and medical professionals" refers to interfaces and communication technologies that allow patients and medical professionals to organize information and communicate it quickly and accurately.

[1681] The present invention relates to a system for effective communication between patients and medical professionals. It is designed to enable patients who speak different languages ​​and elderly people to smoothly communicate and understand information in medical settings. The system is realized using the following main hardware and software:

[1682] Hardware and software used

[1683] Server: A high-performance backend server for handling databases, generative AI models, and encryption technologies.

[1684] Device: Computers and smart devices (e.g., PCs, tablets, smartphones) used by patients and healthcare professionals.

[1685] Generative AI models: Use OpenAI GPT-4 or similar language models to translate medical information.

[1686] Speech recognition engine: A technology that converts speech to text, such as the Google Cloud Speech-to-Text API.

[1687] Database: A relational database such as MySQL or PostgreSQL to manage user information and a medical dictionary.

[1688] Front-end technology: Frameworks for creating user interfaces, such as React and Vue.js.

[1689] Program processing flow

[1690] The system consists of the following steps:

[1691] 1. User Registration and Authentication

[1692] 1. The user (patient) accesses the system and enters the required information such as name, contact information, password, etc. The entered information is sent to the server using a secure communication protocol (SSL / TLS).

[1693] 2. The server stores the received user information in a database and generates a unique identifier (user ID).

[1694] 3. The server generates authentication information and manages the user account. This authentication information is used for subsequent logins.

[1695] 2. Identifying language preferences and needs

[1696] 1. After logging in, the user selects the language they want to use on the language settings screen. For example, they can choose English as their primary language. They can also specify whether they need medical translation.

[1697] 2. The terminal sends the selected language setting information to the server, and the server loads the appropriate medical terminology dictionary and medical database based on that information.

[1698] 3. Loading the Medical Terminology Database

[1699] 1. The server loads an open-source medical dictionary (e.g., MedDRA or SNOMED CT) according to the user's language settings.

[1700] 2. The server integrates the loaded dictionaries and databases into a generative AI model, enabling the translation of specialized medical information.

[1701] 4. Real-time translation of conversations

[1702] 1. The user (patient) enters symptoms and questions by voice or text during a consultation. For example, the user might enter "I have been experiencing severe headaches for the past week."

[1703] 2. The device sends the voice data to the server, which uses a speech recognition engine to convert the voice into text.

[1704] 3. The server uses the generative AI model to translate the text data into the selected language (e.g., Japanese) in real time.

[1705] 4. The terminal displays the translation result on the screen of the user (medical professional) and the patient. For example, the translation may read, "I have been experiencing a severe headache since last week."

[1706] 5. Check and complete the translation result

[1707] 1. The user (medical professional) checks the displayed translation and makes corrections or additions as necessary. For example, adding an additional question such as "When did it start?"

[1708] 2. The terminal sends the corrected information back to the server, and the server performs a retranslation.

[1709] 6. Data Protection and Security

[1710] 1. The server encrypts all patient information and medical data using standard encryption techniques such as AES.

[1711] 2. The server protects data using a secure communication protocol (SSL / TLS) and achieves encrypted communication using public and private keys.

[1712] 3. The device is configured to ensure secure data communications.

[1713] 7. Cultural Sensitivity

[1714] 1. The server applies training data from cultural experts to a generative AI model, implementing an algorithm that produces translations that reflect cultural nuances.

[1715] 2. The generative model will incorporate cultural considerations where necessary, for example, translating with consideration for expressions and etiquette specific to Japanese culture.

[1716] Specific examples

[1717] Scenario: An English-speaking foreign patient visits a Japanese hospital

[1718] 1. The user (patient) accesses the system and registers as a user.

[1719] 2. The device sends the registration information to the server, which stores the information in a database.

[1720] 3. The user (patient) selects "English" in the language settings, and the device sends that information to the server.

[1721] 4. The server loads an English-Japanese medical terminology dictionary and integrates it into the generative AI model.

[1722] 5. During a consultation, the user (patient) speaks, "I have been experiencing severe headaches for the past week."

[1723] 6. The device sends the voice data to the server, and the generative AI model translates it into Japanese as, "I've been experiencing a severe headache since last week."

[1724] 7. The terminal displays the translation results on the user's (medical professional's) screen.

[1725] 8. The user (medical professional) reviews the translation and asks any additional questions.

[1726] 9. The server encrypts all data and uses secure communication protocols.

[1727] 10. The generative model appropriately reflects cultural sensitivity where necessary.

[1728] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1729] Step 1:

[1730] A user accesses the system and enters the necessary information on the account registration screen, such as their name, contact information, and password. This input data is collected as the user's personal information.

[1731] Output: The entered user information is formatted as data.

[1732] Step 2:

[1733] The terminal sends the entered user information to the server, using a secure communication protocol such as SSL / TLS to ensure that the user can send data safely.

[1734] Output: The formatted user information is sent to the server as encrypted data.

[1735] Step 3:

[1736] The server stores the received user information in a database. A unique identifier (user ID) is generated for each user and authentication information is managed. This authentication information is used for subsequent logins.

[1737] Output: User information stored in the database and the generated user ID.

[1738] Step 4:

[1739] After logging in, the user selects the language to use on the language setting screen. They also specify whether they need medical translation. For example, they select English as their primary language and specify that they need translation.

[1740] Output: Selected language settings and translation needs.

[1741] Step 5:

[1742] The device sends the selected language preference information to the server, packaged in a format such as JSON and transmitted over a secure protocol.

[1743] Output: Language preference information is sent to the server.

[1744] Step 6:

[1745] The server loads the appropriate medical dictionary and medical database based on the received language setting information, for example, an English-Japanese medical dictionary.

[1746] Output: Loaded medical dictionary and database.

[1747] Step 7:

[1748] The server then integrates the loaded medical terminology dictionaries and databases into the generative AI model, passing the dictionary data as input to the model to ensure that it can translate specialized medical information.

[1749] Output: A generative AI model integrated with a medical terminology dictionary.

[1750] Step 8:

[1751] During the consultation, the user (patient) can input symptoms and questions by voice or text. For example, they can input "I have been experiencing severe headaches for the past week."

[1752] Output: Audio or text data.

[1753] Step 9:

[1754] The device sends the input voice and text data to the server. In the case of voice data, it is first converted into text using a speech recognition engine, for example, using the Google Cloud Speech-to-Text API.

[1755] Output: The audio data in text format is sent to the server.

[1756] Step 10:

[1757] The server uses a generative AI model to translate the received text data into the selected language (e.g., Japanese) in real time, for example, translating "I have been experiencing severe headaches for the past week" to "I have been experiencing severe headaches since last week."

[1758] Output: Real-time translated text data.

[1759] Step 11:

[1760] The device displays the translation results on the screen of the user (medical professional) and patient, and uses a front-end framework (such as React or Vue.js).

[1761] Output: The translation results are displayed on the screens of the medical professional and the patient.

[1762] Step 12:

[1763] The user (medical professional) checks the displayed translation and makes corrections or additions as necessary, for example adding the question "When did it start?"

[1764] Output: The modified or added text data.

[1765] Step 13:

[1766] The device then sends the corrected information back to the server, again packaged in JSON format or similar and transmitted over a secure protocol.

[1767] Output: The modified text data is sent to the server.

[1768] Step 14:

[1769] The server then uses the generative AI model to perform re-translation, translating the corrected text data back into the local language and providing accurate information to the user.

[1770] Output: The corrected translation text data.

[1771] Step 15:

[1772] The server encrypts all patient information and medical data and protects the data using secure communication protocols (SSL / TLS), specifically using encryption techniques such as AES.

[1773] Output: Encrypted user data and securely stored information.

[1774] Step 16:

[1775] The generative AI model applies training data provided by cultural experts to build algorithms that translate with cultural sensitivity, allowing translations to reflect cultural nuances, such as taking into account expressions and etiquette specific to Japanese culture.

[1776] Output: Culturally sensitive translation results.

[1777] (Application example 1)

[1778] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1779] When a language barrier exists between patients and medical professionals, it can be difficult for patients who speak different languages ​​or for the elderly to smoothly communicate and understand information in medical settings. Communication regarding the language settings of food menus is also a challenge. This can hinder smooth access to medical services and food delivery services.

[1780] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1781] In this invention, the server includes means for receiving a patient's language preference, means for translating medical information based on a generative model, means for displaying the translation result in real time, means for a medical professional to review and correct the translation result, means for protecting the medical information, means for applying a culturally sensitive algorithm, means for receiving a food menu's language preference, means for translating the food information based on a generative model, and means for displaying the translation result to restaurant staff, thereby enabling accurate communication of information between patients and food delivery service users in different language environments.

[1782] A "patient" is a person receiving medical services.

[1783] "Language settings" refers to settings that specify the language used by the user.

[1784] A "generative model" is an artificial intelligence that generates and translates text and speech based on large amounts of data and algorithms.

[1785] "Medical information" refers to all medical information, such as a patient's health condition, diagnosis results, and treatment plan.

[1786] A "translation result" is the result of a generative model converting input text or speech into another language.

[1787] "Display in real time" means that translated text or audio is displayed immediately.

[1788] "Healthcare professional" refers to a professional person engaged in healthcare, such as a doctor, nurse, or pharmacist.

[1789] "Protection" refers to measures taken to prevent unauthorized access or leakage of data.

[1790] A "culturally sensitive algorithm" is an algorithm that takes cultural background into account when producing translations and adjusts the content to be appropriate.

[1791] "Food Menu" means a list of foods and beverages that a Customer can order.

[1792] "Restaurant staff" refers to people who work in a restaurant and take orders and serve food.

[1793] This invention is a system that enables accurate communication between patients and food delivery service users, even in different language environments. The system receives the user's language settings, translates the information based on a generative model, and displays the translation results in real time.

[1794] Hardware and Software Configuration

[1795] The following hardware and software are used to implement this system:

[1796] Hardware: Smartphones, servers

[1797] Software: Firebase, Google Translate API, SpeechRecognition library

[1798] Details of data processing and data calculation

[1799] 1. A user uses a smartphone to input voice or text, such as when the user complains of a symptom, "I have been experiencing severe headaches," or when the user orders a dish, "I would like to order a spicy ramen."

[1800] 2. The device converts the voice data into text data using the SpeechRecognition library.

[1801] 3. The server receives the data sent from the device and translates it based on the generative model. The translation process uses the Google Translate API.

[1802] 4. The translation results are displayed in real time on the user's or staff's device. The displayed information is encrypted using a secure communication protocol to protect the data.

[1803] 5. The server applies a culturally sensitive algorithm and makes any necessary adjustments to the translation result, thereby providing a translation that is appropriate for the user's cultural background.

[1804] Specific example explanation

[1805] Scenario: English-speaking foreign patient

[1806] 1. The user (foreign patient) uses the smartphone app to set their language setting to "English."

[1807] 2. The device sends the registration information to the server, which loads the English-Japanese medical terminology dictionary.

[1808] 3. During a consultation, the user speaks, "I have been experiencing severe headaches."

[1809] 4. The device converts the voice data into text and sends it to the server.

[1810] 5. The server uses the generative model to translate the sentence into Japanese: "I've been experiencing severe headaches since last week."

[1811] 6. The device displays the translation results on the medical professional's screen, and the medical professional asks any additional questions.

[1812] Scenario: An English-speaking customer uses a Japanese food delivery service.

[1813] 1. The user accesses the app and sets the language setting to "English."

[1814] 2. The device sends the information to the server, which loads the English-Japanese menu database.

[1815] 3. The user says, "I would like to order a spicy ramen."

[1816] 4. The device converts the voice data into text and sends it to the server.

[1817] 5. The server uses the generative model to translate "I would like to order spicy ramen" into Japanese.

[1818] 6. The device displays the translation results on the restaurant staff's screen, and the staff asks questions to confirm the order.

[1819] Prompt Sentence Examples

[1820] "Translate the following text from English to Japanese: 'I would like to order a spicy ramen.'"

[1821] By using this system, patients who speak different languages ​​and food delivery service users can overcome communication barriers and receive services smoothly.

[1822] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1823] Step 1:

[1824] A user accesses the app using their smartphone and selects their language preference. As input, the user enters their language preference and sends it to the server. As output, the server receives the information and loads the appropriate terminology database.

[1825] Step 2:

[1826] The device receives voice data input from the user and converts it into text data. This process uses the SpeechRecognition library. The input is voice data, and the output is the converted text data.

[1827] Step 3:

[1828] The terminal sends the converted text data to the server. There is text data as input and data sent to the server as output.

[1829] Step 4:

[1830] The server translates the received text data using a generative model. This translation process uses the Google Translate API. The input is text data, and the output is the translated text.

[1831] Step 5:

[1832] The server sends the translated text to the user's and staff's terminals in real time. The input is the translation result, and the output is the translation result displayed on the terminal.

[1833] Step 6:

[1834] The user (medical professional or staff) can check the displayed translation result and make corrections or ask additional questions if necessary. The input is the translation result, and the output is the corrected information or additional questions.

[1835] Step 7:

[1836] The server encrypts medical information and food menu data and uses a secure communication protocol to protect the data. User and staff input data is used as input, and secure data communication is used as output.

[1837] Step 8:

[1838] The server applies a culturally sensitive algorithm to the generative model and adjusts the translation results as needed. The input is the translation results and cultural data, and the output is a culturally appropriate translation result.

[1839] Step 9:

[1840] The user (patient or consumer) can use their smartphone to check the translation results and facilitate the service (medical consultation or food order).The input is the adjusted translation result, and the output is the user's understanding and next action.

[1841] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1842] This system is designed to facilitate effective communication between patients and medical professionals, and is particularly designed to enable patients who speak different languages ​​and the elderly to smoothly communicate and understand information in medical settings. Furthermore, by incorporating an emotion engine that recognizes the user's emotions, this system allows the translation results to reflect the user's emotional state, enabling more appropriate and sensitive communication.

[1843] The program of this system mainly consists of the following steps:

[1844] User Registration and Authentication

[1845] 1. User: Accesses the system and enters name, contact information, password, etc. on the account registration screen.

[1846] 2. Terminal: Sends the entered registration information to the server.

[1847] 3. Server: Stores the submitted information in a database and generates authentication information that the user will use the next time they log in.

[1848] Verifying your language preferences and needs

[1849] 1. User: After logging in, select the language you want to use on the language setting screen. For example, select "English."

[1850] 2. Terminal: Sends the selected language setting information to the server.

[1851] 3. Server: Based on the received language settings, loads the appropriate medical dictionary and medical database.

[1852] Loading the medical terminology database

[1853] 1. Server: Loads appropriate medical dictionaries and medical databases and integrates them into the generative model, enabling professional medical information translation.

[1854] Real-time conversation translation

[1855] 1. User (Patient): During the consultation, the patient enters symptoms and questions by voice or text (e.g., "I have been experiencing severe headaches for the past week").

[1856] 2. Terminal: Sends input voice and text data to the server.

[1857] 3. Server: The generative model translates the received data into the specified language (e.g., Japanese) (e.g., "I have been experiencing severe headaches since last week").

[1858] 4. Terminal: The translation results are displayed in real time on the screens of the user (medical professional) and the patient.

[1859] Check and complete translation results

[1860] 1. User (medical professional): Checks the displayed translation and makes corrections or completions as necessary, for example, checking for additional symptoms.

[1861] 2. Terminal: Send the corrected information back to the server.

[1862] 3. Server: The generative model re-translates the corrected data and generates the final translation result.

[1863] Information Protection and Security

[1864] 1. Server: All patient information and medical data is encrypted and protected using secure communication protocols.

[1865] 2. Device: Configured to ensure secure data communications.

[1866] Cultural sensitivity

[1867] 1. Server: Implements the algorithm that applies training data from cultural experts to the generative model and produces culturally sensitive translations.

[1868] 2. Generative model: Reflects cultural considerations when they are necessary for the translation results.

[1869] Emotion recognition

[1870] 1. Users: When describing symptoms or questions by voice or text, they may also express emotions (e.g., "I'm very worried about my severe headaches").

[1871] 2. Terminal: Sends emotion data along with voice and text data to the server.

[1872] 3. Server: The emotion engine recognizes the user's emotional state from speech and text and applies the analysis results to the generative model.

[1873] 4. Generative model: Appropriately reflects emotional information in the translation results, producing translations that convey more emotional nuance.

[1874] Specific examples

[1875] Scenario: An English-speaking foreign patient visits a Japanese hospital

[1876] 1. The user (patient) accesses the system and registers as a user.

[1877] 2. The device sends the registration information to the server, which stores the information in a database.

[1878] 3. The user (patient) selects "English" in the language settings, and the device sends that information to the server.

[1879] 4. The server loads the English-Japanese medical terminology dictionary and integrates it into the generative model.

[1880] 5. During a consultation, the user (patient) speaks, "I have been experiencing severe headaches for the past week. I'm very worried about it."

[1881] 6. The device sends the emotion data along with the voice data to the server, and the emotion engine analyzes the emotion "worried."

[1882] 7. Based on the data, the server translates the sentence into Japanese as "I have been experiencing severe headaches since last week and I am very worried about it."

[1883] 8. The terminal displays the translation results on the user's (medical professional's) screen.

[1884] 9. The user (medical professional) reviews the translation results and asks any additional questions or clarifications.

[1885] 10. The server encrypts all data and uses secure communication protocols.

[1886] 11. The generative model appropriately reflects cultural sensitivity where necessary.

[1887] This system will enable elderly and foreign patients to receive medical care smoothly and ensure the accuracy and reliability of medical information. In addition, by combining it with an emotion engine, more detailed communication will be possible.

[1888] The processing flow will be explained below.

[1889] Step 1:

[1890] User: Accesses the system and enters name, contact information, password, etc. on the account registration screen.

[1891] Step 2:

[1892] Terminal: Sends the entered registration information to the server.

[1893] Step 3:

[1894] Server: Stores the submitted information in a database and generates authentication information that the user will use the next time they log in.

[1895] Step 4:

[1896] User: After logging in, select the language to use on the language setting screen. For example, select "English."

[1897] Step 5:

[1898] Terminal: Sends the selected language setting information to the server.

[1899] Step 6:

[1900] Server: Loads the appropriate medical dictionary and medical database based on the selected language settings.

[1901] Step 7:

[1902] Server: Prepares for translation by integrating the loaded medical dictionary and medical database into the generative model.

[1903] Step 8:

[1904] User (Patient): During the consultation, the patient will voice or text their symptoms and questions (e.g., "I have been experiencing severe headaches for the past week. I'm very worried about it").

[1905] Step 9:

[1906] Terminal: Sends input voice and text data to the server.

[1907] Step 10:

[1908] Server: The emotion engine recognizes the emotion "worried" from speech and text, and applies the analysis results to the generative model.

[1909] Step 11:

[1910] Server: The generative model translates the received data into the specified language (e.g., Japanese) (e.g., "I've been experiencing severe headaches since last week. I'm very worried about it").

[1911] Step 12:

[1912] Terminal: Displays the translation results in real time on the user's (medical professional's) screen.

[1913] Step 13:

[1914] User (medical professional): Checks the displayed translation and makes corrections or completions as necessary, for example, checking for additional symptoms.

[1915] Step 14:

[1916] Terminal: Send the corrected information back to the server.

[1917] Step 15:

[1918] Server: The generative model re-translates the corrected data and generates the final translation result.

[1919] Step 16:

[1920] Terminal: displays the final translation results on the screen of the user (patient) and medical professional.

[1921] Step 17:

[1922] Server: Protects patient personal information and medical data by encrypting it and applying secure communication protocols.

[1923] Step 18:

[1924] Server: Implements the algorithm that applies training data provided by cultural experts to the generative model and performs translation with cultural sensitivity.

[1925] Step 19:

[1926] Generative model: Properly reflecting the content of translations that require cultural sensitivity.

[1927] In this way, the system of the present invention creates an environment where elderly and foreign patients can receive medical care smoothly, ensuring the accuracy and reliability of medical information. In addition, by combining it with an emotion engine, more detailed communication is possible.

[1928] Example 2

[1929] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1930] In today's medical environment, patients who speak different languages ​​and the elderly face difficulties in communicating appropriately in medical settings. Furthermore, simple translation without considering the patient's emotional state makes it difficult for medical professionals to understand the patient's true needs and feelings. Furthermore, confidentiality of medical information and cultural sensitivity are also important, but there is still a lack of systems that can adequately address these issues.

[1931] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1932] In this invention, the server includes means for receiving the patient's language setting, means for recognizing the patient's emotional state, means for translating medical information based on a generative model, means for displaying the translation results in real time, means for a medical professional to review and correct the translation results, means for protecting the medical information, and means for applying a culturally sensitive algorithm, thereby enabling smooth communication between patients and elderly people who speak different languages ​​in medical settings and realizing sensitive and accurate medical information transmission that takes into account the patient's emotional state and cultural background.

[1933] The "means for receiving language settings" is a function that allows the user to specify the language they want to use and transmit that information to the server.

[1934] The "means for recognizing emotional states" is a function that analyzes and identifies emotions contained in the user's statements and inputs.

[1935] "Means for translating medical information based on a generative model" refers to a function that uses a generative AI model to translate user utterances or input into a specified language.

[1936] "Means for displaying translation results in real time" refers to a function that immediately outputs the translated content to a display device.

[1937] "Means for medical professionals to check and correct translation results" is a function that allows medical professionals to check the displayed translation results and correct them if necessary.

[1938] "Measures for protecting medical information" refers to the ability to encrypt patient information and medical data and protect the confidentiality of data using secure communication protocols.

[1939] "Means for applying a culturally sensitive algorithm" refers to a function that applies an algorithm that takes cultural background into account and makes appropriate adjustments to the translation results.

[1940] The present invention provides a system that enables patients and elderly people who speak different languages ​​to smoothly communicate and understand information in medical settings. The system includes a means for receiving a patient's language preference, a means for recognizing an emotional state, a means for translating medical information based on a generative model, a means for displaying the translation results in real time, a means for a medical professional to review and correct the translation results, a means for protecting medical information, and a means for applying a culturally sensitive algorithm.

[1941] User Registration and Authentication

[1942] 1. The user accesses the system and enters their name, contact information, and password on the account registration screen. The device sends the entered registration information to the server. The server stores the received information in a database and generates a user ID and authentication token. This authentication token is used the next time the user logs in.

[1943] Verifying your language preferences and needs

[1944] 1. After logging in, the user selects the language they want to use on the language setting screen. For example, they select "English." The device sends this language setting information to the server. Based on the selected language, the server loads the appropriate medical terminology dictionary and medical database and integrates them into the generative model.

[1945] Loading the medical terminology database

[1946] 1. The server integrates medical terminology dictionaries and medical databases into the generative model to enable the translation of specialized medical information. This process ensures accurate translation of specialized terminology and medical-related information.

[1947] Real-time conversation translation

[1948] 1. During a consultation, the user (patient) inputs their symptoms or questions by voice or text. For example, they might input "I have been experiencing severe headaches for the past week." The device sends the input voice or text data to the server. The server uses a generative model to translate the received data into the specified language (for example, Japanese). The translation result is "I have been experiencing severe headaches since last week." The device displays the translation result in real time on the screens of the user (medical professional) and the patient.

[1949] Check and complete translation results

[1950] 1. The user (medical professional) checks the displayed translation results and makes corrections or additions as necessary. The device then sends the corrected information back to the server, where the generative model re-translates the corrected data to generate the final translation results.

[1951] Information Protection and Security

[1952] 1. The server encrypts all patient information and medical data and protects the data using secure communication protocols. The device is configured to ensure secure data communication.

[1953] Cultural sensitivity

[1954] 1. The server applies training data provided by cultural experts to a generative model, implementing an algorithm that achieves culturally sensitive translation. The generative model then reflects any cultural considerations that may be required in the translation results.

[1955] Emotion recognition

[1956] 1. When a user inputs symptoms or questions using voice or text, they also express their emotions. For example, they might input "I'm very worried about my severe headaches." The device sends the emotion data along with the voice and text data to the server. The server's emotion engine analyzes the user's emotional state from the voice or text and applies the analysis results to a generative model. The generative model then incorporates the emotion information into the translation results, resulting in a translation that conveys more emotional nuance.

[1957] Specific examples

[1958] Scenario: An English-speaking foreign patient visits a Japanese hospital

[1959] 1. The user (patient) accesses the system, enters their name, contact information, and password on the registration screen, enters the required information in the form, and clicks the submit button.

[1960] 2. The device sends the registration information to the server, after which a registration success message is displayed.

[1961] 3. The server saves the information to a database and generates an authentication token. The user ID and token are registered in the database.

[1962] 4. The user (patient) logs in and selects "English" in the language settings. Select English from the drop-down menu and click the Save button.

[1963] 5. The device sends the language setting information to the server.

[1964] 6. The server loads the English-Japanese medical terminology dictionary and integrates it into the generative model.

[1965] 7. During a consultation, the user (patient) dictates, "I have been experiencing severe headaches for the past week. I'm very worried about it." This can be done using a dedicated microphone or a text input field.

[1966] 8. The device sends the voice data and emotion data to the server, and the emotion engine analyzes the emotion "worried."

[1967] 9. Based on the data, the server translates the sentence into Japanese: "I have been experiencing severe headaches since last week and I am very worried about it."

[1968] 10. The device displays the translation results on the screen of the user (medical professional) and patient. The translation results are displayed in real time on the screen of a smartphone or tablet.

[1969] 11. The user (medical professional) checks the translation results and asks any additional questions or clarifications. Checks the content displayed on the tablet and enters additional instructions if necessary.

[1970] 12. The server encrypts all data and uses secure communication protocols.

[1971] 13. The generative model appropriately reflects cultural sensitivity where necessary.

[1972] Specific prompt examples

[1973] "In a scenario where an English-speaking foreign patient is seen at a hospital in Japan, and the patient says, 'I have been experiencing severe headaches for the past week. I'm very worried about it,' how would this system work?"

[1974] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1975] Step 1:

[1976] A user accesses the system and enters their name, contact information, and password on the account registration screen. The device sends the entered registration information to the server. The server stores the received information in a database and generates a user ID and authentication token.

[1977] Input: Name, Contact, Password

[1978] Output: User ID, authentication token

[1979] Specific operation: The user enters information into the form and clicks the submit button. The device sends the information to the server, which stores it in the database.

[1980] Step 2:

[1981] The user logs in and selects the language to use on the language setting screen. The terminal sends the language setting information to the server. The server loads the medical terminology dictionary and medical database based on the selected language.

[1982] Input: Login information, language setting information

[1983] Output: Medical terminology dictionary, medical database

[1984] Specific operation: A user logs in, selects the language they want to use from a drop-down menu, and clicks the Save button. The device sends the language setting information to the server, which loads the appropriate database.

[1985] Step 3:

[1986] The server integrates medical terminology dictionaries and medical databases into the generative model, preparing it for the translation of specialized medical information.

[1987] Input: Medical terminology dictionary, medical database

[1988] Output: A unified generative model

[1989] Specific operation: The server loads the medical terminology dictionary and medical database and performs the process of integrating them into the generative model.

[1990] Step 4:

[1991] During a consultation, the user (patient) inputs symptoms and questions by voice or text. The device sends the input voice or text data to the server, where a generative model translates the data and converts it into the specified language.

[1992] Input: Audio data, text data

[1993] Output: Translation data

[1994] Specific operation: The user speaks into the microphone or types into the text field. The device sends the data to the server, which then translates it using the generative model.

[1995] Step 5:

[1996] The terminal displays the translation results in real time on the screen of the user (medical professional) and the patient.

[1997] Input: Translation data

[1998] Output: Translation results displayed on the screen

[1999] Specific operation: The translation results are displayed in real time on the screen of your smartphone or tablet.

[2000] Step 6:

[2001] The user (medical professional) checks the displayed translation results and makes corrections or additions as necessary. The device then sends the corrected information back to the server, where the generative model re-translates the corrected data to generate the final translation results.

[2002] Input: Corrected translation data

[2003] Output: Final translation data

[2004] How it works: A medical professional reviews the translation results, enters corrections on the tablet, and the device sends the data back to the server, which then uses the generative model to re-translate the corrected data.

[2005] Step 7:

[2006] The server encrypts all patient information and medical data and protects the data using secure communication protocols.

[2007] Input: Patient information, medical data

[2008] Output: Encrypted data, secure communication

[2009] What happens: The server encrypts the data using an encryption algorithm and establishes a secure communication protocol.

[2010] Step 8:

[2011] The server implements an algorithm that applies training data provided by cultural experts to a generative model to achieve culturally sensitive translation.

[2012] Input: Training data

[2013] Output: Culturally sensitive algorithm

[2014] Specific operation: The server applies the training data to the generative model and updates the algorithm.

[2015] Step 9:

[2016] When users input symptoms or questions by voice or text, they also express their emotions. The device sends the emotion data along with the voice and text data to the server. The server's emotion engine analyzes the user's emotional state from the voice or text and applies the analysis results to a generative model. The generative model reflects the emotion information in the translation results, resulting in a translation that conveys more emotional nuance.

[2017] Input: Voice data, text data, emotion data

[2018] Output: Translation data reflecting emotional information

[2019] How it works: The user inputs their emotions through voice or text. The device sends the data to the server, which then uses an emotion engine to recognize the emotional state and apply it to the generative model.

[2020] (Application example 2)

[2021] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2022] The problem that this invention aims to solve is to support patients and elderly people who speak different languages ​​in smoothly communicating and understanding information in medical settings, and to realize more appropriate communication through emotion recognition. Another problem is to achieve both data protection and cultural considerations. Furthermore, it is expected that the same effect will be achieved in virtual stores as an application.

[2023] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2024] In this invention, the server includes means for receiving a patient's language setting, means for translating medical information based on a generative model, means for displaying the translation result in real time, means for a medical professional to check and correct the translation result, means for protecting the medical information, means for applying a culturally sensitive algorithm, and means for recognizing emotional states using an emotion engine and reflecting them in the translation result, thereby enabling smooth and appropriate communication between patients, elderly people, and even virtual store customers who speak different languages.

[2025] The "means for receiving patient language setting" is an interface for the patient to select or set the language they use and send that information to the system.

[2026] "Means for translating medical information based on a generative model" means a function that uses a generative AI model to translate specialized medical information into different languages ​​based on received language preferences.

[2027] "Means for displaying translation results in real time" refers to a display or user interface that instantly displays translated medical information so that patients and medical professionals can quickly review it.

[2028] "Means for medical professionals to review and correct translation results" refers to a function that allows medical professionals to review the translation results displayed by the system and correct or supplement them as necessary.

[2029] "Measures to protect medical information" refers to the use of encryption and secure communication protocols to protect patient privacy and safely handle medical information.

[2030] "Means for applying culturally sensitive algorithms" are algorithms that use training data that takes into account cross-cultural sensitivities and that reflect cultural context in translated medical information.

[2031] "Means of recognizing emotional states using an emotion engine and reflecting them in the translation results" is a function that analyzes the emotions contained in the patient's voice or text and appropriately reflects that emotional state in the translation results.

[2032] The present invention is implemented as a system for facilitating communication between patients and medical professionals who speak different languages, and can also support communication between customers and staff in virtual stores.

[2033] The system mainly consists of a server, a terminal, and a user. In this embodiment, the system is implemented using terminals such as smartphones, smart glasses, and head-mounted displays, and uses generative AI models and related algorithms on the server.

[2034] Overall system flow

[2035] 1. User Registration and Authentication

[2036] The user accesses the system and registers an account by entering their name, contact information, password, etc.

[2037] The device sends the entered information to the server, which stores it in a database and generates authentication information to use for subsequent logins.

[2038] 2. Identifying language preferences and needs

[2039] After logging in, the user sets the language to be used, for example, English or Japanese.

[2040] The terminal transmits the selected language information to the server, which then loads the corresponding medical terminology dictionary and product database.

[2041] 3. Real-time translation of conversations

[2042] Users (patients or customers) input questions or symptoms by voice or text. For example, a voice input might say, "I have been experiencing severe headaches for the past week."

[2043] The device sends the input data to the server, which then translates it using a generative AI model (using TensorFlow), and displays the translation results on the device in real time.

[2044] In addition, an emotion engine (using the Azure Emotion API) analyzes emotions and reflects them in the translation results.

[2045] 4. Check and complete the translation result

[2046] The user (medical professional or staff) reviews the translation and corrects it if necessary.

[2047] The device then sends the corrected information back to the server, and the final translation result is generated.

[2048] 5. Data Protection and Security

[2049] The server encrypts all data and protects it with secure communication protocols (SSL / TLS).

[2050] This allows for smooth and appropriate communication with patients and customers who speak different languages, and the system is also culturally sensitive, supporting more natural communication.

[2051] Specific examples

[2052] Use in medical settings

[2053] Patient (English speaker): "I have been experiencing severe headaches for the past week. I'm very worried about it."

[2054] The device sends voice and emotion data to the server, which translates it into "I've been experiencing severe headaches since last week and I'm very worried about it," and displays the results reflecting the emotion.

[2055] Use in virtual stores

[2056] Customer (English speaker): "What ingredients are in this skincare product? I have sensitive skin and I'm really worried about potential reactions."

[2057] The device sends voice and emotion data to the server, which translates it into "What ingredients are in this skin care product? I have sensitive skin, so I'm worried about a reaction," and displays the results reflecting the emotion.

[2058] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2059] Step 1:

[2060] A user accesses the system and enters their name, contact information, password, etc. on the account registration screen. The device sends this information to the server, which then stores it in a database. Authentication information to be used for subsequent logins is also generated.

[2061] Input: Name, Contact, Password

[2062] Data processing: Save the input information in the database and generate authentication information

[2063] Output: Credentials

[2064] Step 2:

[2065] After logging in, the user selects the language they want to use. The terminal sends the selected language information to the server, which then loads the corresponding medical terminology dictionary and product database based on that information, thereby preparing the necessary translation resources.

[2066] Input: Select your language

[2067] Data processing: Load medical dictionary and product database based on language settings

[2068] Output: Available translation resources

[2069] Step 3:

[2070] The user (patient or customer) inputs a question or symptom via voice or text. For example, a voice input might say, "I have been experiencing severe headaches for the past week." The device sends this to the server, which then uses a generative AI model to translate the input data.

[2071] Input: Voice or text input

[2072] Data transformation: Translating input data using generative AI models

[2073] Output: Translation result

[2074] Step 4:

[2075] The device receives the voice and text data sent to the server and uses an emotion engine to recognize the emotional state. Using this emotion data, the server reflects the emotional information in the translation results.

[2076] Input: Voice and text data

[2077] Data processing: Emotion recognition by emotion engine and reflection in translation results

[2078] Output: Translation results that reflect emotional information

[2079] Step 5:

[2080] The translation results are displayed on the device in real time. Users (medical professionals and staff) can review the translation results and make corrections as needed. The corrected information is then sent back to the server, where the final translation results are generated.

[2081] Input: Translation results that reflect emotional information

[2082] Data processing: User checks and corrects, and final translation results are generated.

[2083] Output: The final translation result

[2084] Step 6:

[2085] All data is encrypted and protected using secure communication protocols (SSL / TLS), ensuring patient and customer information is kept secure.

[2086] Input: Translation results and user information

[2087] Data processing: Encrypting data and using secure communication protocols

[2088] Output: Secure data

[2089] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2090] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2091] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2092] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2093] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2094] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2095] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2096] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2097] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2098] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2099] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2100] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2101] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2102] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2103] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2104] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2105] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2106] As an example of a system configured with a sin...

Claims

1. a means for receiving a patient's language preference; a means for translating medical information based on a generative model; A means to display translation results in real time; A means for medical professionals to review and correct translation results; measures to protect medical information; means of applying culturally sensitive algorithms; A system including:

2. The system of claim 1 , wherein the generative model performs translation by integrating a medical terminology dictionary and a medical database.

3. 10. The system of claim 1, wherein the medical information is protected by encrypting the data and using a secure communication protocol.

Images