System

Abstract

A system is provided.SOLUTION: A system comprising: means for unblocking a generative model and initializing a model for user-specific speech generation based on an organized database comprising a speech history for each user; means for converting speech input from a user to text and sending the text to a server; means for generating an appropriate response based on the text received by the server using the generative model; means for using speech synthesis technology to audibly output the generated response to the user; and means for collecting feedback from the user and storing the feedback in a database for use in improving future speech generation.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] As the elderly population increases, the risk of developing dementia increases. Dementia prevention is a challenge, especially for elderly people living alone, who have few opportunities to converse and whose brains are not stimulated. Furthermore, hospitals and nursing homes are facing serious staff shortages, making it difficult to secure sufficient time for conversation. For this reason, there is a need to provide an environment where elderly people can easily enjoy conversation and promote brain activity. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system that utilizes a generative model to enable natural conversations with elderly people. Specifically, the system includes: means for disabling the generative model and initializing a model for user-specific conversation generation based on organized data including each user's conversation history; means for converting speech input from the user into text and sending the text to a server; means for generating appropriate responses using the generative model based on the text received by the server; means for using speech synthesis technology to output the generated responses to the user; and means for collecting feedback from the user and storing the feedback in a database for use in improving future conversation generation. The system also includes context analysis means for analyzing the user's utterances and extracting the conversational intent, and learning means for continuously improving the generative model using the user feedback data. This allows elderly people to easily enjoy conversations and promotes brain activation, thereby contributing to the prevention of dementia.

[0006] A "generative model" is a machine learning algorithm that uses natural language processing to generate appropriate responses to input text.

[0007] "User-specific conversation history" refers to a record of past conversations that a particular user has had, which can be used as context information for future conversations.

[0008] "Voice input" refers to words or phrases spoken by a user and the act of inputting these into a terminal as digital signals.

[0009] "Convert to text" refers to the process of converting speech input into written information, thereby making it in a machine-parseable format.

[0010] A "server" refers to a computer system that has data processing and storage functions and receives and processes requests from clients (terminals) via a network.

[0011] "Context analysis" refers to the process of analyzing the content of text or speech entered by a user to understand their intent and related information.

[0012] "Speech synthesis technology" refers to the general technology for converting text information into speech and outputting it in a natural-sounding format.

[0013] "Feedback" refers to evaluations and opinions about the system's response provided by users, and is used as data for improving the system.

[0014] A "database" refers to a system that accumulates digital information and stores it in a format that can be managed and referenced, and stores various data including feedback and conversation history.

[0015] "Learning method" refers to the process or algorithm used to improve and update the generative model using collected data. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The embodiment of the present invention relates to a conversation service for preventing dementia that utilizes a generative model, and the specific program processing and explanation in natural language are provided below.

[0038] An embodiment of a conversation system using a generative model

[0039] The purpose of this system is to provide an environment where elderly people can enjoy daily conversations and promote brain activation. The main components and operation details are shown below.

[0040] Server Processing

[0041] 1. Initializing the AI ​​model

[0042] The server first loads the generative AI model, reads each user's conversation history, and initializes the model based on that information. This enables personalized responses that reflect the user's past conversation data.

[0043] Example: The server initialized the generative AI model based on the conversation history of user ID 123.

[0044] 2. Receiving and analyzing user requests

[0045] When a user sends voice input from a device, the server receives it, converts it into text, and performs context analysis to understand the user's intent and requests.

[0046] Example: The server receives the text "How's the weather today?" and interprets it as a question about the weather.

[0047] 3. Generating responses using a generative model

[0048] Based on the analysis results, the server uses a generative AI model to generate an appropriate response that provides the best answer to the user's question.

[0049] Example: The server uses a generative AI model to generate the response, "Today's weather is sunny and the temperature is 25 degrees."

[0050] 4. Gather feedback and improve the model

[0051] After the conversation is over, the server receives feedback from the user and stores it in a database, which helps improve the quality of the generative AI model.

[0052] Example: The server stores the feedback "the response was helpful" in its database.

[0053] Terminal handling

[0054] 1. Voice to Text

[0055] The device receives the user's voice and converts it into text using voice recognition technology.

[0056] Example: A user says "What's the weather?" which is converted by the device to "What's the weather?"

[0057] 2. Sending a text

[0058] The terminal sends the converted text to the server.

[0059] Example: The device sends the text "Tell me the weather" to the server.

[0060] 3. Receiving a response from the server and outputting audio

[0061] The device receives the response text from the server and converts it internally into speech using speech synthesis technology, which is then played back to the user.

[0062] Example: The device spoke the response to the user: "Today's weather is sunny and the temperature is 25 degrees."

[0063] User Roles

[0064] 1. Start a conversation

[0065] The user starts a conversation by speaking to the device, asking specific questions or making requests.

[0066] Example: A user says, "Alexa, what's the weather like today?"

[0067] 2. Response acceptance and feedback

[0068] The user listens to the responses provided by the device and provides appropriate feedback, which is used to improve the quality of future conversations.

[0069] Example: The user provides feedback saying, "Thanks, now I can go for a walk."

[0070] Specific conversation examples

[0071] User: "Alexa, what's the weather forecast for this week?"

[0072] Terminal: "Send a request to the server."

[0073] Server: Received text from user ID 456 asking "What's the weather forecast this week?" and used the generative model to generate the response "There will be many sunny days this week, with a maximum temperature of 25 degrees."

[0074] Device: "There will be many sunny days this week, with a high of 25 degrees," the user is told.

[0075] User: "Thanks, that's very helpful."

[0076] This concludes the description of an embodiment of a conversation system using a generative model. This system allows elderly people to enjoy conversations with others on a daily basis, promoting brain activity and contributing to the prevention of dementia.

[0077] The processing flow will be explained below.

[0078] Step 1:

[0079] The server initializes the generative model. Specifically, it reads the conversation history of each user, customizes the generative model based on that data, and then initializes it.

[0080] Step 2:

[0081] The user verbally inputs questions and requests into the device in a natural conversational style.

[0082] Step 3:

[0083] The device receives the user's voice and converts it into text data using voice recognition technology. If the voice input is "Tell me the weather," the text will also be "Tell me the weather."

[0084] Step 4:

[0085] The terminal sends the converted text to the server, which also includes identification information such as the user ID.

[0086] Step 5:

[0087] The server receives the text data sent from the device, analyzes the received data, and performs context analysis to understand the user's intent.

[0088] Step 6:

[0089] The server uses the generative model to generate an appropriate response based on the analysis results, such as "Today's weather is sunny and the temperature is 20 degrees."

[0090] Step 7:

[0091] The server generates a response and sends it back to the terminal in text format, which contains the response content.

[0092] Step 8:

[0093] The terminal generates voice data based on the received response text using speech synthesis technology and outputs the voice to the user.

[0094] Step 9:

[0095] The user receives a voice response from the terminal and provides feedback as needed, such as "Thank you, that was helpful."

[0096] Step 10:

[0097] The terminal transmits the feedback from the user back to the server, and the transmitted data includes the content of the user's feedback.

[0098] Step 11:

[0099] The server analyzes the received feedback data and stores it in a database for future improvements to the generative model. This feedback data is used as reference information for future conversation generation.

[0100] Example 1

[0101] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0102] To contribute to the prevention of dementia among the elderly, it is important to provide an environment where people can enjoy everyday conversations. However, providing conversations that meet individual needs requires advanced natural language processing technology and responses that reflect the user's past conversation history. Furthermore, to improve the quality of the generated responses, a system is needed to efficiently collect user feedback and use it to improve the model. However, current technology does not provide a system that can consistently achieve these goals.

[0103] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0104] In this invention, the server includes means for loading a generative AI model and reading each user's conversation history to initialize the model, means for converting voice input from the user into text and sending the text to the server, means for the server to perform context analysis of the received text and generate an appropriate response using the generative AI model, means for using speech synthesis technology to output the generated response to the user, and means for collecting feedback from users and storing the feedback in a database for use in improving future conversation generation, thereby providing conversations tailored to individual needs and enabling continuous improvement of the model using the feedback.

[0105] A "generative AI model" is a machine learning model trained to generate text and data using artificial intelligence (AI).

[0106] "Conversation history" is a record of past conversations that a particular user has had, data that can be used to personalize future responses.

[0107] "Speech recognition" is the technology that analyzes voice data and converts it into corresponding text.

[0108] "Context analysis" is a natural language processing technique used to understand the content and intent of input text.

[0109] "Speech synthesis technology" is a technology that converts text data into speech and generates artificially synthesized speech.

[0110] "Feedback" refers to ratings and comments on responses provided by users, and is information used to improve the system.

[0111] A "database" is a collection of information that stores data in an organized manner and can be efficiently accessed when needed.

[0112] "Natural language processing technology" is technology for understanding, interpreting, and generating human language.

[0113] "Machine learning" is a technique that uses data to improve algorithms for pattern recognition and prediction.

[0114] This invention relates to a conversation system for dementia prevention using a generative AI model. In particular, it provides a mechanism for initializing the generative model based on an individual conversation history, converting speech input into text, generating responses using the generative model, synthesizing speech output, and collecting and learning feedback.

[0115] Overall system configuration

[0116] This system mainly consists of three main components: a server, a terminal, and a user. The server has the generative AI model, the terminal is responsible for speech recognition and speech synthesis, and the user has a natural conversation with the system. Each component is described in detail below.

[0117] Server Roles

[0118] Loading and initializing the generative AI model

[0119] The server loads a generative AI model (e.g., GPT-4) and runs the model using a high-performance GPU. Each user's conversation history is retrieved from a database and input into the model for initialization. This enables personalized responses that reflect the user's past conversation data.

[0120] Receiving and analyzing user requests

[0121] The server receives voice input from the device and converts it to text using a speech recognition engine (e.g., Google Speech-to-Text API). The converted text is then contextually analyzed using a generative AI model to understand the user's intent.

[0122] Response generation using generative models

[0123] Based on the analysis results, the server uses a generative AI model to generate an appropriate response to the user's question or request. In this process, the user's question is given as an input prompt, and the model generates an appropriate answer.

[0124] Example: In response to a user question, "What's the weather forecast this week?", the generative AI model generates the response, "There will be many sunny days this week, with a maximum temperature of 25 degrees."

[0125] Gathering feedback and improving the model

[0126] After the conversation, the server receives feedback from the user and stores it in a database. The collected feedback is used to improve the quality of the generative AI model and for continuous learning.

[0127] Device Role

[0128] Voice to text conversion

[0129] The device collects the user's voice using a built-in microphone and converts it into text using voice recognition technology (e.g., Google Speech-to-Text API). This conversion process ensures that the system accurately conveys natural speech from the user.

[0130] Sending text and voice output

[0131] The device sends the converted text to the server and receives a response from the server. The received response text is converted into speech using speech synthesis technology (e.g., Amazon Polly) and conveyed to the user.

[0132] Example: The device will speak the response to the user, "There will be many sunny days this week, with a maximum temperature of 25 degrees."

[0133] User Roles

[0134] Initiating a conversation and accepting a response

[0135] The user initiates a conversation by speaking into the device, asking specific questions or making requests, listening to the responses provided by the device and providing feedback as needed.

[0136] For example, when a user says, "Assistant, what's the weather like today?", the system responds by providing an appropriate response.

[0137] Specific examples

[0138] 1. User: "Assistant, what's the weather forecast for this week?"

[0139] 2. Device: Converts speech to text and sends it to the server.

[0140] 3. Server: Input the text "What's the weather forecast this week?" into the generative AI model and generate a response.

[0141] 4. Server: Sends the response "There will be many sunny days this week, and the maximum temperature will be 25 degrees" to the terminal.

[0142] 5. Terminal: Converts the received text into speech and conveys it to the user.

[0143] 6. User: "Thanks, that would be great."

[0144] In this way, elderly people can enjoy conversation with someone on a daily basis, which can promote brain activation and contribute to the prevention of dementia.

[0145] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0146] Step 1: Load and initialize the AI ​​model

[0147] ------

[0148] The server loads the generative AI model and reads the conversation history for each user.

[0149] Input: The server retrieves a file from the database containing the user ID, the corresponding conversation history, and the generative AI model (e.g., GPT-4).

[0150] process:

[0151] The server loads the generative AI model using a high-performance GPU.

[0152] Next, we load the past conversation history from the database based on the user ID and use this history to initialize the model.

[0153] Output: The initialized generative AI model runs in memory and reflects the conversational context of each user.

[0154] Specific behavior:

[0155] The server loads the GPT-4 model into memory on a GPU server in the data center.

[0156] The server retrieves the past conversation log of user ID 123 from the database and incorporates it into the model.

[0157] Step 2: Convert voice to text

[0158] ------

[0159] The terminal converts the user's voice into text and sends it to the server.

[0160] Input: Voice data spoken by the user into the device.

[0161] process:

[0162] The device's built-in microphone collects the user's voice and converts it into text using a local or cloud-based speech recognition engine (e.g., Google Speech-to-Text API).

[0163] Output: Text data is generated and sent from the terminal to the server.

[0164] Specific behavior:

[0165] When a user says, "Alexa, what's the weather today?", the device records this speech and converts it into text: "Alexa, what's the weather today?"

[0166] Step 3: Contextual analysis of the text

[0167] ------

[0168] The server parses the received text and uses a generative AI model to generate an appropriate response.

[0169] Input: User's text data received by the server from the device.

[0170] process:

[0171] The server provides the generative AI model with the user's text as an input prompt, which the model analyzes internally and generates an appropriate response.

[0172] Natural language processing (NLP) technology is used for context analysis.

[0173] Output: You get the generated response text.

[0174] Specific behavior:

[0175] The server receives the text "What's the weather forecast this week?" and analyzes it to generate a response saying "There will be many sunny days this week, with a maximum temperature of 25 degrees."

[0176] Step 4: Text-to-speech response

[0177] ------

[0178] The terminal receives the response text from the server, converts it into voice, and conveys it to the user.

[0179] Input: Response text data sent from the server.

[0180] process:

[0181] The device uses speech synthesis technology (e.g., Amazon Polly) to convert the received text data into speech.

[0182] Output: Audio data is generated and played through the speaker.

[0183] Specific behavior:

[0184] The device converts the response "There will be many sunny days this week, with a maximum temperature of 25 degrees" into speech and conveys it to the user.

[0185] Step 5: Gather feedback

[0186] ------

[0187] The server stores user feedback in a database to help improve the quality of the model.

[0188] Input: Feedback data provided by the user.

[0189] process:

[0190] The server receives the feedback and stores it in a database.

[0191] The collected feedback is used to retrain or tune the generative AI model.

[0192] Output: The saved feedback data.

[0193] Specific behavior:

[0194] The user provides feedback such as "Thanks, now I can go for a walk," and the server stores this in a database, which is then used to improve the model.

[0195] This is the specific program processing flow of this system. This allows elderly people to enjoy conversations with others on a daily basis, promoting brain activity and helping to prevent dementia.

[0196] (Application example 1)

[0197] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0198] In modern society, cognitive decline among the elderly is a serious problem. In particular, lack of communication inhibits brain activation and contributes to the accelerated progression of dementia. However, many elderly people have limited opportunities to enjoy conversation with others on a daily basis, and effective methods for providing such opportunities are needed. Even community facilities and cafes for the elderly lack systems that provide consistent, high-quality conversation partners. Technology is needed to solve this problem and improve the quality of life for the elderly.

[0199] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0200] In this invention, the server includes means for disabling the generative model and initializing a model for user-specific conversation generation based on organized data including each user's conversation history, means for converting speech input from the user into text and sending the text to the server, means for generating an appropriate response using the generative model based on the text received by the server, means for using speech synthesis technology to output the generated response to the user, means for collecting feedback from the user and storing the feedback in a database for use in improving future conversation generation, and means for providing natural conversation with users at a community facility for the elderly using a smart device linked to the generative model. This allows elderly people to enjoy daily conversations even in community facilities and cafes, thereby activating their brains.

[0201] A "generative model" is an algorithm that uses artificial intelligence to generate text, images, etc.

[0202] "Conversation history for each user" refers to a record of conversations that each user has had in the past.

[0203] A "means for initializing a model" is a method for starting the operation of a generative model based on specific conditions or data.

[0204] "Means for converting voice input to text" refers to technology that converts the voice spoken by a user into text information.

[0205] "Means for transmitting to the server" refers to a method for transmitting data from the terminal to the server.

[0206] The "means for generating an appropriate response" is a technology for generating a response for forming a natural conversation based on a user's input.

[0207] "Means using speech synthesis technology" refers to technology for converting text data into speech and outputting it.

[0208] "Feedback collection methods" are methods for collecting user-provided ratings and opinions.

[0209] "Means for storing data in a database" refers to the technology for recording collected data so that it can be used later.

[0210] "Community facilities" are social institutions used by the elderly to enjoy social interactions and activities.

[0211] "Smart terminal" is a general term for highly functional portable devices, including smartphones and tablets.

[0212] "Natural conversation" refers to interaction with artificial intelligence that is similar to a human conversation.

[0213] "Brain activation" refers to stimulation and activities that enhance brain function.

[0214] This invention is a conversation support system for elderly community facilities that utilizes generative models. The aim is to provide an environment where elderly people can enjoy daily conversations and promote brain activation. The main components and operation are detailed below.

[0215] Server Processing

[0216] Initializing the AI ​​model

[0217] The server first loads the generative AI model, reads each user's conversation history, and initializes the model based on that. This enables personalized responses that reflect the user's past conversation data. An example of a generative AI model used is OpenAI's GPT-3.

[0218] Receiving and analyzing user requests

[0219] When a user sends voice input from their device, the server receives it and performs context analysis on the converted text. This allows the server to understand the user's intent and requests. The specific software used for the voice recognition technology is the Google Speech-to-Text API.

[0220] Generating responses using generative models

[0221] Based on the analysis results, the server uses a generative AI model to generate an appropriate response, which provides the best answer to the user's question.

[0222] Gathering feedback and improving the model

[0223] After the conversation is over, the server receives feedback from the user and stores it in a database, which helps improve the quality of the generative AI model.

[0224] Terminal handling

[0225] Voice to text conversion

[0226] The device receives the user's voice and converts it into text. This process uses voice recognition technology. Specific hardware used is a smart speaker (e.g., Amazon Echo) or a tablet (e.g., iPad).

[0227] Sending a text

[0228] The terminal sends the converted text to the server using a communication protocol such as HTTPS.

[0229] Receiving a response from the server and outputting audio

[0230] The device receives the response text from the server, converts it internally into speech using speech synthesis technology, and uses the Google Text-to-Speech API to play the converted speech for the user.

[0231] User Roles

[0232] Start a conversation

[0233] The user starts a conversation by speaking to the device, asking specific questions or making requests. For example, "What's the weather like today?" or "Tell me the latest news."

[0234] Response acceptance and feedback

[0235] The user listens to the responses provided by the device and provides appropriate feedback, which is used to improve the quality of future conversations.

[0236] Specific conversation examples

[0237] User: "What kind of music do you like today?"

[0238] Terminal: "Send a request to the server."

[0239] Server: Received text from user ID 123 asking, "What kind of music do you like today?" and used the generative model to generate the response, "I've been enjoying classical music lately. I think Beethoven's symphonies are particularly wonderful."

[0240] Device: "I've been enjoying classical music lately. I think Beethoven's symphonies are particularly wonderful."

[0241] User: "Thanks, that's very helpful."

[0242] This concludes the description of an embodiment of a conversation support system for elderly community facilities that uses a generative model. This system allows elderly people to enjoy conversations with others on a daily basis, promoting brain activation and contributing to the prevention of dementia.

[0243] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0244] Step 1:

[0245] Receiving voice input

[0246] Subject: Device

[0247] How it works: The device receives audio spoken by the user.

[0248] Input: User speech (e.g., "What kind of music do you like today?")

[0249] Data processing and calculation: The microphone installed on the device converts the voice data into a digital signal.

[0250] Output: Digital audio data

[0251] Step 2:

[0252] Converting audio data to text

[0253] Subject: Device

[0254] How it works: The device converts digital voice data into text.

[0255] Input: Digital audio data

[0256] Data processing and calculation: The device's voice recognition software (e.g., Google Speech-to-Text API) converts the voice data into text information.

[0257] Output: Text data (e.g., "What kind of music do you like today?")

[0258] Step 3:

[0259] Sending text data to the server

[0260] Subject: Device

[0261] Operation: The terminal sends the converted text data to the server.

[0262] Input: Text data

[0263] Data processing and calculation: Text data is sent to the server using the HTTPS protocol.

[0264] Output: Request data to the server

[0265] Step 4:

[0266] Contextual analysis of text data

[0267] Subject: Server

[0268] How it works: The server analyzes the received text data and extracts the user's intent and requests.

[0269] Input: Text data

[0270] Data processing and calculation: The server's context analysis engine analyzes the text data and identifies the intent and request.

[0271] Output: Context analysis results (e.g., identifying that the question is about music)

[0272] Step 5:

[0273] Generating an appropriate response

[0274] Subject: Server

[0275] How it works: The server generates a response using a generative AI model (e.g., OpenAI GPT-3) based on the contextual analysis results.

[0276] Input: Context analysis results

[0277] Data processing and calculation: The generative AI model generates prompt sentences and generates optimal responses.

[0278] Output: Response text data (e.g., "I've been enjoying classical music lately. I think Beethoven's symphonies are particularly wonderful.")

[0279] Step 6:

[0280] Terminal reception of response data

[0281] Subject: Device

[0282] Operation: The terminal receives the response text data sent from the server.

[0283] Input: Response text data

[0284] Data processing and calculation: The terminal receives the response data using the HTTPS protocol.

[0285] Output: Response text data (e.g., "I've been enjoying classical music lately. I think Beethoven's symphonies are particularly wonderful.")

[0286] Step 7:

[0287] Response text-to-speech

[0288] Subject: Device

[0289] Operation: The device converts the received response text data into speech.

[0290] Input: Response text data

[0291] Data processing and calculation: The device's speech synthesis software (e.g., Google Text-to-Speech API) converts the text data into voice data.

[0292] Output: Response audio data

[0293] Step 8:

[0294] Response voice output to the user

[0295] Subject: Device

[0296] Action: The device plays the audio data and communicates it to the user.

[0297] Input: Response audio data

[0298] Data processing and calculation: Plays audio through the device's built-in speaker.

[0299] Output: Response speech (e.g., "I've been enjoying classical music lately. I think Beethoven's symphonies are particularly wonderful.")

[0300] Step 9:

[0301] Gathering feedback

[0302] Subject: Device

[0303] Action: The device receives feedback from the user.

[0304] Input: User feedback

[0305] Data processing and calculation: The device converts voice input back into text or collects feedback via touch input, etc.

[0306] Output: Feedback text data

[0307] Step 10:

[0308] Feedback data sent to server and stored

[0309] Subject: Terminal and Server

[0310] Operation: The device sends the feedback text data to the server, which stores it in a database.

[0311] Input: Feedback text data

[0312] Data processing and calculation: The terminal sends the feedback text data to the server, and the server stores the received feedback data in a database.

[0313] Output: Stored feedback data

[0314] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0315] The embodiment of the present invention relates to a conversation service for preventing dementia that utilizes a generative model and an emotion engine, and the specific program processing and explanation in natural language are provided below.

[0316] An embodiment of a conversation system using a generative model and an emotion engine

[0317] The system aims to provide an environment where elderly people can enjoy daily conversations and promote the prevention of dementia through brain activation and emotion recognition. The main components and operation details are shown below.

[0318] Server Processing

[0319] 1. Initializing the AI ​​model

[0320] The server loads the generative AI model, reads each user's conversation history, and initializes the model based on that data, enabling personalized responses that reflect the user's past conversation data.

[0321] Example: The server initialized the generative AI model based on the conversation history of user ID 123.

[0322] 2. Receiving user requests and analyzing their sentiment

[0323] When a user sends a voice input from a terminal, the server receives it, converts it into text, analyzes the content, and simultaneously analyzes the user's emotions using an emotion engine to obtain emotion information.

[0324] Example: The server receives the text "How's the weather today?" and then uses an emotion engine to recognize the emotion "happy" from the user's voice.

[0325] 3. Generating responses using a generative model

[0326] Based on the analysis results and emotional information, the server uses a generative AI model to generate an appropriate response that reflects the user's current emotions.

[0327] Example: The server used a generative AI model to generate the response, "Today's weather is sunny and the temperature is 20 degrees. It looks like it's going to be a great day!"

[0328] 4. Gather feedback and improve the model

[0329] After the conversation is over, the server receives user feedback and emotion data and stores it in a database. The collected data will be used to improve the quality of the generative AI model and emotion engine.

[0330] Example: The server stores the feedback "the response was helpful" and the emotion data "fun" in a database.

[0331] Terminal handling

[0332] 1. Voice to Text

[0333] The device receives the user's voice and converts it into text data using voice recognition technology. If the voice input is "Tell me the weather," the text will also be "Tell me the weather."

[0334] Example: A user says "What's the weather?" which is converted by the device to "What's the weather?"

[0335] 2. Sending a text

[0336] The terminal sends the converted text to the server, which also includes identification information such as the user ID.

[0337] Example: The device sends the text "Tell me the weather" to the server.

[0338] 3. Receiving a response from the server and outputting audio

[0339] The device receives the response text from the server and converts it internally into speech using speech synthesis technology, which is then played back to the user.

[0340] Example: The device spoke the response to the user: "Today's weather is sunny and the temperature is 20 degrees. It looks like it's going to be a great day!"

[0341] User Roles

[0342] 1. Start a conversation

[0343] The user starts a conversation by speaking to the terminal, asking specific questions or making requests verbally.

[0344] Example: A user says, "Alexa, what's the weather like today?"

[0345] 2. Response acceptance and feedback

[0346] The user listens to the responses provided by the device and provides appropriate feedback, which is used to improve the quality of future conversations.

[0347] Example: The user provides feedback saying, "Thanks, now I can go for a walk."

[0348] Specific conversation examples

[0349] User: "Alexa, what's the weather forecast for this week?"

[0350] Terminal: "Send a request to the server."

[0351] Server: Received text from user ID 456 asking "What's the weather forecast this week?" and used the emotion engine to recognize "expectation" from the user's voice. Using the generative model, generated the response: "There will be many sunny days this week, with a maximum temperature of 25 degrees. Looking forward to it!"

[0352] Device: "There will be lots of sunny days this week, with a high of 25 degrees. Looking forward to it!"

[0353] User: "Thanks, that's very helpful."

[0354] This concludes the description of an embodiment of a conversation system using a generative model and an emotion engine. This system allows elderly people to enjoy conversations with others on a daily basis, and contributes to the prevention of dementia through brain activation and emotion recognition.

[0355] The processing flow will be explained below.

[0356] Step 1:

[0357] The server initializes the generative model. Specifically, it reads the conversation history of each user, customizes the generative model based on that data, and then initializes it.

[0358] Step 2:

[0359] The user verbally inputs questions and requests into the device in a natural conversational style.

[0360] Step 3:

[0361] The device receives the user's voice and converts it into text data using voice recognition technology. For example, if the voice input is "Tell me the weather," the text will also be "Tell me the weather."

[0362] Step 4:

[0363] The terminal sends the converted text to the server, which also includes identification information such as the user ID.

[0364] Step 5:

[0365] The server receives the text data sent from the device, analyzes the received data, and performs context analysis to understand the user's intent.

[0366] Step 6:

[0367] The server uses an emotion engine to extract the user's emotion from the speech along with context analysis. The emotion engine analyzes the tone and rate of the speech to identify the user's emotion.

[0368] Step 7:

[0369] The server uses a generative model to generate an appropriate response based on the analysis and emotional information, for example, taking into account a question about the weather and the emotion "fun" to tailor the response.

[0370] Step 8:

[0371] The server sends the generated response back to the terminal in text format. The response content is included in the sent data.

[0372] Step 9:

[0373] The device uses speech synthesis technology to generate voice data based on the received response text and outputs it to the user. For example, it might output, "Today's weather is sunny, and the temperature is 20 degrees. It looks like it's going to be a nice day today!"

[0374] Step 10:

[0375] The user receives a voice response from the terminal and provides feedback as needed, such as "Thank you, that was helpful."

[0376] Step 11:

[0377] The terminal transmits the feedback from the user to the server, and the transmitted data includes the content of the user's feedback.

[0378] Step 12:

[0379] The server analyzes the received feedback and emotion data and stores it in a database for future use in improving the generative model and emotion engine.

[0380] Example 2

[0381] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0382] Brain activation and emotional recognition through everyday conversations are important for preventing dementia in the elderly. However, it is difficult for elderly people to find suitable conversation partners on a daily basis, and conventional conversation systems are insufficient in providing individualized responses that reflect the user's emotions and past conversation history, limiting their effectiveness in preventing dementia. The present invention solves these problems and provides technology that promotes dementia prevention while allowing elderly people to enjoy everyday conversations.

[0383] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for loading a generative AI model and initializing a user-specific model based on data including each user's conversation history; means for converting voice input from the user into text and sending the text to the server; means for analyzing the text received by the server and acquiring emotional information using an emotion engine; means for generating a response based on the analysis result and emotional information using the generative AI model; means for outputting the generated response to the user by voice using speech synthesis technology; and means for collecting feedback from the user and storing the feedback and emotional data in a database. This allows elderly people to enjoy individually tailored conversations on a daily basis, and promotes dementia prevention through brain activation and emotion recognition.

[0384] A "generative AI model" is an artificial intelligence-based model used to generate conversations with users, utilizing natural language processing techniques.

[0385] "Voice input" refers to the voice that the user utters into the terminal, and is used to convert this into text data and send it to the server.

[0386] "Text-to-text" refers to the process of converting voice input into text data, which is done using speech recognition technology.

[0387] "Emotion engine" refers to software or functionality for analyzing and acquiring emotional information from the user's speech and voice.

[0388] "Response generation" refers to the process of using a generative AI model to create an appropriate response to the user based on the acquired text data and emotional information.

[0389] "Speech synthesis technology" refers to technology for converting generated response text into voice data and providing it to the user aloud.

[0390] "Conversation history for each user" is a record of past conversations with a specific user, and is used to personalize the generative AI model.

[0391] "Feedback" refers to opinions and thoughts provided by users regarding a conversation, and is used to improve the system.

[0392] "Database" refers to an information storage system for saving and managing conversation history, feedback, emotional data, etc. with users.

[0393] MODE FOR CARRYING OUT THE INVENTION

[0394] The embodiment of this invention relates to a conversation service for dementia prevention that utilizes a generative model and an emotion engine, and aims to provide an environment where elderly people can enjoy conversation on a daily basis, and promote dementia prevention through brain activation and emotion recognition. Specific program processing and details of the embodiment are described below.

[0395] Server Processing

[0396] The server first loads a generative AI model (e.g., GPT-3). Next, it reads each user's conversation history from a database and initializes the generative AI model based on that data. This initialization enables personalized responses that reflect the user's past conversation data.

[0397] When a user sends voice input from a device, the server converts the voice data into text and receives it. At the same time, it uses an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the user's emotions and obtain emotional information. Based on the analysis results and emotional information, the server uses a generative AI model to generate an appropriate response and sends the response in text format to the device.

[0398] After the conversation is over, the server stores the user's feedback and emotional data in a database, which can later be used to improve the generative AI model and emotion engine.

[0399] Terminal handling

[0400] The device receives the user's voice input and converts it into text data using voice recognition technology (e.g., Google Cloud Speech-to-Text). The converted text data is then sent from the device to a server. This text data also includes identification information such as the user ID.

[0401] When the device receives the response text from the server, it uses speech synthesis technology (e.g., Amazon Polly) to convert the received text into speech and provides it to the user.

[0402] User Roles

[0403] The user initiates a conversation by speaking to the device. At this time, they can verbally express specific questions or requests. They listen to the voice responses provided by the device and provide feedback as needed. This feedback is used to improve the quality of future conversations.

[0404] Specific conversation examples

[0405] User: "Assistant, what's the weather forecast for this week?"

[0406] Terminal: "Send a request to the server."

[0407] Server: Received text from user ID 456 asking "What's the weather forecast this week?" and used the emotion engine to recognize "expectation" from the user's voice. Using the generative AI model, generated the response: "There will be many sunny days this week, with a maximum temperature of 25 degrees. Looking forward to it!"

[0408] Device: "There will be lots of sunny days this week, with a high of 25 degrees. Looking forward to it!"

[0409] User: "Thanks, that's very helpful."

[0410] This concludes the description of an embodiment of a conversation system using a generative model and an emotion engine. This system allows elderly people to enjoy conversations with others on a daily basis, and contributes to the prevention of dementia through brain activation and emotion recognition.

[0411] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0412] Program processing flow

[0413] Server Processing

[0414] Step 1: Initializing the generative AI model

[0415] How it works: The server first loads a generative AI model (e.g., GPT-3) that is used to reflect the user's unique past conversation history.

[0416] Input: Conversation history data for each user retrieved from the database.

[0417] Data processing and calculation: The server analyzes conversation history data and applies it to a generative AI model to customize it for each user.

[0418] Output: An initialized generative AI model.

[0419] Step 2: Receiving voice input and converting it to text

[0420] What happens: The device receives the user's voice input and converts it into text using speech recognition technology (e.g., Google Cloud Speech-to-Text).

[0421] Input: User's voice input.

[0422] Data processing and data calculation: Speech to text conversion using speech recognition technology.

[0423] Output: Text data (e.g. "Tell me the weather").

[0424] Step 3: Send a text

[0425] Specific operation: The terminal sends the converted text data along with the user ID to the server.

[0426] Input: Text data and user ID.

[0427] Data processing and data calculation: Formation of data structures for sending text data and identification information to the server.

[0428] Output: The text data sent to the server and the user ID.

[0429] Step 4: Text and sentiment analysis

[0430] Specific operation: The server analyzes the received text data and obtains the user's emotional information using an emotion engine (e.g., IBM Watson Tone Analyzer).

[0431] Input: Received text data.

[0432] Data processing and data calculation: Sentiment analysis and text content analysis using emotion engines.

[0433] Output: Analysis results and user sentiment information.

[0434] Step 5: Generate a response

[0435] How it works: The server uses a generative AI model to generate an appropriate response based on the analysis results and emotional information.

[0436] Input: Analysis results and emotion information.

[0437] Data processing and data calculation: Response generation using generative AI models.

[0438] Output: Response text (e.g. "Today the weather is sunny and the temperature is 20 degrees. It looks like it's going to be a great day!").

[0439] Step 6: Sending a Response

[0440] Specific operation: The server sends the generated response text to the terminal.

[0441] Input: The response text.

[0442] Data processing and data calculation: Transfer of response text.

[0443] Output: Sending the response text from the server to the terminal.

[0444] Step 7: Transcribing and outputting responses

[0445] Specific operation: The device converts the received response text into speech using speech synthesis technology (e.g., Amazon Polly) and provides it to the user.

[0446] Input: The response text.

[0447] Data processing and data arithmetic: Response text to speech conversion.

[0448] Output: A spoken response (e.g. "Today the weather is sunny and the temperature is 20 degrees. It's going to be a great day!").

[0449] Step 8: Collect and store feedback

[0450] Specific operation: After the conversation ends, the server stores the user's feedback and emotion data in a database.

[0451] Input: User feedback and sentiment data.

[0452] Data processing and data computation: Analysis and storage of feedback and emotion data.

[0453] Output: Feedback and sentiment data stored in a database.

[0454] User Action

[0455] Step 1: Start a conversation

[0456] Specific actions: The user speaks to the device and verbally communicates a specific question or request.

[0457] Input: User's voice.

[0458] Data processing and data arithmetic: Shaping what we say.

[0459] Output: Voice input to the device (e.g., "Assistant, what's the weather today?").

[0460] Step 2: Acceptance of responses and feedback

[0461] Specific operation: The user listens to the voice response from the device and provides appropriate feedback based on the content.

[0462] Input: A voice response from the device.

[0463] Data processing and data arithmetic: Shaping feedback based on response content.

[0464] Output: User feedback (e.g. "Thanks, that's very helpful").

[0465] These are the specific processing steps of this system's program. This allows elderly people to enjoy conversations with others on a daily basis, and contributes to the prevention of dementia through brain activation and emotional recognition.

[0466] (Application example 2)

[0467] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0468] To enable elderly people to use food delivery services smoothly, simple voice-activated operation is required. However, conventional systems have difficulty responding to individual emotions, resulting in a poor user experience. Furthermore, they are unable to generate personalized suggestions or responses based on order history, which may lead to a decrease in user satisfaction. There is a need to provide a system that can solve these issues and enable elderly people to use food delivery services comfortably.

[0469] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0470] In this invention, the server includes means for disabling the generative model and initializing a user-specific model for conversation generation based on organized data including each user's conversation history, means for converting speech input from the user into text and sending the text to the server, means for generating appropriate responses using the generative model based on the text received by the server, means for using speech synthesis technology to output the generated responses to the user, means for collecting feedback from the user and storing the feedback in a database for use in improving future conversation generation, means for using an emotion engine to ensure that suggested responses reflect the user's emotional state, and means for suggesting appropriate menu items to the user based on emotions and past ordering history. This allows for the generation of personalized responses based on the user's emotions, enabling elderly people to smoothly use food delivery services in an emotionally satisfying manner.

[0471] 1. A "generative model" is a machine learning algorithm that generates appropriate responses based on user input data.

[0472] 2. "Conversation history" is a record of past conversations a user has had, and is data used to provide personalized support in future conversations.

[0473] 3. "Voice input" refers to voice data uttered by a user through a device such as a microphone.

[0474] 4. "Convert to text" is the process of converting voice input into text data.

[0475] 5. “Server” means a computer system that hosts Generative Models and processes data received from Users.

[0476] 6. An "emotion engine" is an algorithm that analyzes a user's emotions from voice and text data and identifies their emotional state.

[0477] 7. "Speech synthesis technology" means technology that converts text data into voice data and is used to communicate generated responses to the user by voice.

[0478] 8. "Feedback" means the responses or opinions provided by a user and any emotional data associated with those responses.

[0479] 9. "Order History" means a record of past orders placed by a User and is used to make recommendations for future orders.

[0480] 10. "Menu Suggestions" are a list of foods or items that the system suggests to the user based on past ordering history and the user's emotional state.

[0481] This invention relates to a food delivery ordering support system for seniors that utilizes a generative model and an emotion engine. The system enables seniors to easily order food delivery using voice and provides personalized suggestions through emotion analysis to enhance the user experience.

[0482] System Overview

[0483] Hardware

[0484] Smartphones: Devices used by seniors that support voice input and output.

[0485] Server: A computer system that hosts the generative model and emotion engine and processes data received from users.

[0486] software

[0487] Generative AI model (GPT-2): A machine learning algorithm used to generate appropriate responses to user input.

[0488] Emotion engine: Software for analyzing user emotions from voice and text data to identify emotional states. For example, we use the EmotionAnalyzer library.

[0489] Speech recognition technology: Technology to convert the elderly's voice input into text. As an example, we will use the Google Speech-to-Text API.

[0490] Speech synthesis technology: Technology to convert responses from the server into voice data and convey them to the user. As an example, we will use the Google Text-to-Speech API.

[0491] System processing flow

[0492] Server Processing

[0493] 1. Model initialization: The server loads the generative model, reads each user's order history, and initializes the model based on that data. This allows for personalized responses that reflect the user's past order data.

[0494] 2. Receiving and analyzing voice input: The terminal receives voice input from the elderly person and analyzes the converted text. At the same time, it uses an emotion engine to analyze the user's emotions and obtain emotional information.

[0495] 3. Response and suggestion generation: Based on the analysis results and sentiment information, a generative model is used to generate appropriate responses and suggestions. For example, suggesting appropriate menu items based on the user's sentiment and past ordering history.

[0496] 4. Feedback collection: After the conversation is over, the server receives feedback and sentiment data from the user and stores it in a database, which can improve the quality of responses in the future.

[0497] Terminal handling

[0498] 1. Voice input and text conversion: The device receives the voice spoken by the elderly person and converts it into text data using voice recognition technology.

[0499] 2. Sending the text data: The converted text data is sent to the server, including identification information such as the user ID.

[0500] 3. Receiving response and outputting voice: Receives the response text from the server, converts it into voice using speech synthesis technology, and plays it back to the user.

[0501] User Roles

[0502] 1. Start the conversation: The elderly person starts speaking to the device and placing an order by voice.

[0503] 2. Response acceptance and feedback: The elderly person listens to the response from the device, confirms the order, and provides feedback. The feedback is used to improve the quality of response generation in future.

[0504] Specific examples

[0505] User: "Can you help me place an order?"

[0506] Terminal: "Of course. What would you like to order today?"

[0507] User: "I want pizza."

[0508] Server: Based on user ID 123's order history, suggest a menu item that matches their order history, saying, "How about pizza? Margherita or pepperoni are also delicious. Which would you like?"

[0509] Terminal: "How about some pizza? Margherita or pepperoni are both delicious. Which would you like?"

[0510] User: "Margherita, please."

[0511] Terminal: "Okay, I'll order a Margherita pizza!"

[0512] Prompt Sentence Examples

[0513] User input: "I want pizza."

[0514] Sentiment analysis result: “Expectation”

[0515] Suggestion generation prompt: "The user has a sense of anticipation. Suggest Margherita pizza or pepperoni pizza based on their past orders."

[0516] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0517] Step 1:

[0518] Initializing the Server

[0519] The server loads the generative model and initializes it with each user's order history, allowing it to generate user-specific responses based on past data.

[0520] Input: User ID and order history data

[0521] Processing: The server analyzes the data for each user and adjusts the generative model

[0522] Output: An initialized generative model

[0523] Step 2:

[0524] Receiving and converting voice input

[0525] The user speaks to the smartphone to input voice, which the device receives and converts into text using voice recognition technology.

[0526] Input: User voice input

[0527] Processing: Using speech recognition technology (e.g., Google Speech-to-Text API) to convert voice data into text data

[0528] Output: Text data

[0529] Step 3:

[0530] Sending text data

[0531] The device sends the converted text data to the server, which also includes identification information such as the user ID.

[0532] Input: Text data and user ID

[0533] Process: Send the text data and user ID to the server

[0534] Output: The input text received by the server

[0535] Step 4:

[0536] Emotion analysis

[0537] The server analyzes the user's emotions using an emotion engine based on the received text data.

[0538] Input: Text data

[0539] Processing: Analyze user emotions using an emotion engine (e.g., EmotionAnalyzer library)

[0540] Output: Emotional state data

[0541] Step 5:

[0542] Generate responses and suggestions

[0543] Generative AI models are used to generate appropriate responses and suggestions based on sentiment information and past order history.

[0544] Input: Text data, emotional state data, order history data

[0545] Processing: Generative AI models generate appropriate responses and menu suggestions based on the input data.

[0546] Output: Response text data

[0547] Step 6:

[0548] Audio output of the response

[0549] The terminal receives the response text data from the server, converts it into voice data using voice synthesis technology, and plays it back to the user.

[0550] Input: Response text data

[0551] Processing: Convert text data into audio data using speech synthesis technology (e.g., Google Text-to-Speech API).

[0552] Output: The audio data to be played back to the user

[0553] Step 7:

[0554] Gathering feedback

[0555] After the conversation is finished, the user provides feedback to the terminal, which sends this feedback to the server, which stores it in a database.

[0556] Input: User feedback

[0557] Processing: The server records the feedback and emotional state data and uses it to improve the next conversation generation.

[0558] Output: Data for an improved generative model

[0559] As a result of the above, the system allows seniors to easily order food delivery using voice commands and provides personalized service through sentiment analysis.

[0560] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0561] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0562] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0563] [Second embodiment]

[0564] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0565] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0566] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0567] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0568] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0569] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0570] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0571] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0572] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0573] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0574] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0575] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0576] The embodiment of the present invention relates to a conversation service for preventing dementia that utilizes a generative model, and the specific program processing and explanation in natural language are provided below.

[0577] An embodiment of a conversation system using a generative model

[0578] The purpose of this system is to provide an environment where elderly people can enjoy daily conversations and promote brain activation. The main components and operation details are shown below.

[0579] Server Processing

[0580] 1. Initializing the AI ​​model

[0581] The server first loads the generative AI model, reads each user's conversation history, and initializes the model based on that information. This enables personalized responses that reflect the user's past conversation data.

[0582] Example: The server initialized the generative AI model based on the conversation history of user ID 123.

[0583] 2. Receiving and analyzing user requests

[0584] When a user sends voice input from a device, the server receives it, converts it into text, and performs context analysis to understand the user's intent and requests.

[0585] Example: The server receives the text "How's the weather today?" and interprets it as a question about the weather.

[0586] 3. Generating responses using a generative model

[0587] Based on the analysis results, the server uses a generative AI model to generate an appropriate response that provides the best answer to the user's question.

[0588] Example: The server uses a generative AI model to generate the response, "Today's weather is sunny and the temperature is 25 degrees."

[0589] 4. Gather feedback and improve the model

[0590] After the conversation is over, the server receives feedback from the user and stores it in a database, which helps improve the quality of the generative AI model.

[0591] Example: The server stores the feedback "the response was helpful" in its database.

[0592] Terminal handling

[0593] 1. Voice to Text

[0594] The device receives the user's voice and converts it into text using voice recognition technology.

[0595] Example: A user says "What's the weather?" which is converted by the device to "What's the weather?"

[0596] 2. Sending a text

[0597] The terminal sends the converted text to the server.

[0598] Example: The device sends the text "Tell me the weather" to the server.

[0599] 3. Receiving a response from the server and outputting audio

[0600] The device receives the response text from the server and converts it internally into speech using speech synthesis technology, which is then played back to the user.

[0601] Example: The device spoke the response to the user: "Today's weather is sunny and the temperature is 25 degrees."

[0602] User Roles

[0603] 1. Start a conversation

[0604] The user starts a conversation by speaking to the device, asking specific questions or making requests.

[0605] Example: A user says, "Alexa, what's the weather like today?"

[0606] 2. Response acceptance and feedback

[0607] The user listens to the responses provided by the device and provides appropriate feedback, which is used to improve the quality of future conversations.

[0608] Example: The user provides feedback saying, "Thanks, now I can go for a walk."

[0609] Specific conversation examples

[0610] User: "Alexa, what's the weather forecast for this week?"

[0611] Terminal: "Send a request to the server."

[0612] Server: Received text from user ID 456 asking "What's the weather forecast this week?" and used the generative model to generate the response "There will be many sunny days this week, with a maximum temperature of 25 degrees."

[0613] Device: "There will be many sunny days this week, with a high of 25 degrees," the user is told.

[0614] User: "Thanks, that's very helpful."

[0615] This concludes the description of an embodiment of a conversation system using a generative model. This system allows elderly people to enjoy conversations with others on a daily basis, promoting brain activity and contributing to the prevention of dementia.

[0616] The processing flow will be explained below.

[0617] Step 1:

[0618] The server initializes the generative model. Specifically, it reads the conversation history of each user, customizes the generative model based on that data, and then initializes it.

[0619] Step 2:

[0620] The user verbally inputs questions and requests into the device in a natural conversational style.

[0621] Step 3:

[0622] The device receives the user's voice and converts it into text data using voice recognition technology. If the voice input is "Tell me the weather," the text will also be "Tell me the weather."

[0623] Step 4:

[0624] The terminal sends the converted text to the server, which also includes identification information such as the user ID.

[0625] Step 5:

[0626] The server receives the text data sent from the device, analyzes the received data, and performs context analysis to understand the user's intent.

[0627] Step 6:

[0628] The server uses the generative model to generate an appropriate response based on the analysis results, such as "Today's weather is sunny and the temperature is 20 degrees."

[0629] Step 7:

[0630] The server generates a response and sends it back to the terminal in text format, which contains the response content.

[0631] Step 8:

[0632] The terminal generates voice data based on the received response text using speech synthesis technology and outputs the voice to the user.

[0633] Step 9:

[0634] The user receives a voice response from the terminal and provides feedback as needed, such as "Thank you, that was helpful."

[0635] Step 10:

[0636] The terminal transmits the feedback from the user back to the server, and the transmitted data includes the content of the user's feedback.

[0637] Step 11:

[0638] The server analyzes the received feedback data and stores it in a database for future improvements to the generative model. This feedback data is used as reference information for future conversation generation.

[0639] Example 1

[0640] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0641] To contribute to the prevention of dementia among the elderly, it is important to provide an environment where people can enjoy everyday conversations. However, providing conversations that meet individual needs requires advanced natural language processing technology and responses that reflect the user's past conversation history. Furthermore, to improve the quality of the generated responses, a system is needed to efficiently collect user feedback and use it to improve the model. However, current technology does not provide a system that can consistently achieve these goals.

[0642] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0643] In this invention, the server includes means for loading a generative AI model and reading each user's conversation history to initialize the model, means for converting voice input from the user into text and sending the text to the server, means for the server to perform context analysis of the received text and generate an appropriate response using the generative AI model, means for using speech synthesis technology to output the generated response to the user, and means for collecting feedback from users and storing the feedback in a database for use in improving future conversation generation, thereby providing conversations tailored to individual needs and enabling continuous improvement of the model using the feedback.

[0644] A "generative AI model" is a machine learning model trained to generate text and data using artificial intelligence (AI).

[0645] "Conversation history" is a record of past conversations that a particular user has had, data that can be used to personalize future responses.

[0646] "Speech recognition" is the technology that analyzes voice data and converts it into corresponding text.

[0647] "Context analysis" is a natural language processing technique used to understand the content and intent of input text.

[0648] "Speech synthesis technology" is a technology that converts text data into speech and generates artificially synthesized speech.

[0649] "Feedback" refers to ratings and comments on responses provided by users, and is information used to improve the system.

[0650] A "database" is a collection of information that stores data in an organized manner and can be efficiently accessed when needed.

[0651] "Natural language processing technology" is technology for understanding, interpreting, and generating human language.

[0652] "Machine learning" is a technique that uses data to improve algorithms for pattern recognition and prediction.

[0653] This invention relates to a conversation system for dementia prevention using a generative AI model. In particular, it provides a mechanism for initializing the generative model based on an individual conversation history, converting speech input into text, generating responses using the generative model, synthesizing speech output, and collecting and learning feedback.

[0654] Overall system configuration

[0655] This system mainly consists of three main components: a server, a terminal, and a user. The server has the generative AI model, the terminal is responsible for speech recognition and speech synthesis, and the user has a natural conversation with the system. Each component is described in detail below.

[0656] Server Roles

[0657] Loading and initializing the generative AI model

[0658] The server loads a generative AI model (e.g., GPT-4) and runs the model using a high-performance GPU. Each user's conversation history is retrieved from a database and input into the model for initialization. This enables personalized responses that reflect the user's past conversation data.

[0659] Receiving and analyzing user requests

[0660] The server receives voice input from the device and converts it to text using a speech recognition engine (e.g., Google Speech-to-Text API). The converted text is then contextually analyzed using a generative AI model to understand the user's intent.

[0661] Response generation using generative models

[0662] Based on the analysis results, the server uses a generative AI model to generate an appropriate response to the user's question or request. In this process, the user's question is given as an input prompt, and the model generates an appropriate answer.

[0663] Example: In response to a user question, "What's the weather forecast this week?", the generative AI model generates the response, "There will be many sunny days this week, with a maximum temperature of 25 degrees."

[0664] Gathering feedback and improving the model

[0665] After the conversation, the server receives feedback from the user and stores it in a database. The collected feedback is used to improve the quality of the generative AI model and for continuous learning.

[0666] Device Role

[0667] Voice to text conversion

[0668] The device collects the user's voice using a built-in microphone and converts it into text using voice recognition technology (e.g., Google Speech-to-Text API). This conversion process ensures that the system accurately conveys natural speech from the user.

[0669] Sending text and voice output

[0670] The device sends the converted text to the server and receives a response from the server. The received response text is converted into speech using speech synthesis technology (e.g., Amazon Polly) and conveyed to the user.

[0671] Example: The device will speak the response to the user, "There will be many sunny days this week, with a maximum temperature of 25 degrees."

[0672] User Roles

[0673] Initiating a conversation and accepting a response

[0674] The user initiates a conversation by speaking into the device, asking specific questions or making requests, listening to the responses provided by the device and providing feedback as needed.

[0675] For example, when a user says, "Assistant, what's the weather like today?", the system responds by providing an appropriate response.

[0676] Specific examples

[0677] 1. User: "Assistant, what's the weather forecast for this week?"

[0678] 2. Device: Converts speech to text and sends it to the server.

[0679] 3. Server: Input the text "What's the weather forecast this week?" into the generative AI model and generate a response.

[0680] 4. Server: Sends the response "There will be many sunny days this week, and the maximum temperature will be 25 degrees" to the terminal.

[0681] 5. Terminal: Converts the received text into speech and conveys it to the user.

[0682] 6. User: "Thanks, that would be great."

[0683] In this way, elderly people can enjoy conversation with someone on a daily basis, which can promote brain activation and contribute to the prevention of dementia.

[0684] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0685] Step 1: Load and initialize the AI ​​model

[0686] ------

[0687] The server loads the generative AI model and reads the conversation history for each user.

[0688] Input: The server retrieves a file from the database containing the user ID, the corresponding conversation history, and the generative AI model (e.g., GPT-4).

[0689] process:

[0690] The server loads the generative AI model using a high-performance GPU.

[0691] Next, we load the past conversation history from the database based on the user ID and use this history to initialize the model.

[0692] Output: The initialized generative AI model runs in memory and reflects the conversational context of each user.

[0693] Specific behavior:

[0694] The server loads the GPT-4 model into memory on a GPU server in the data center.

[0695] The server retrieves the past conversation log of user ID 123 from the database and incorporates it into the model.

[0696] Step 2: Convert voice to text

[0697] ------

[0698] The terminal converts the user's voice into text and sends it to the server.

[0699] Input: Voice data spoken by the user into the device.

[0700] process:

[0701] The device's built-in microphone collects the user's voice and converts it into text using a local or cloud-based speech recognition engine (e.g., Google Speech-to-Text API).

[0702] Output: Text data is generated and sent from the terminal to the server.

[0703] Specific behavior:

[0704] When a user says, "Alexa, what's the weather today?", the device records this speech and converts it into text: "Alexa, what's the weather today?"

[0705] Step 3: Contextual analysis of the text

[0706] ------

[0707] The server parses the received text and uses a generative AI model to generate an appropriate response.

[0708] Input: User's text data received by the server from the device.

[0709] process:

[0710] The server provides the generative AI model with the user's text as an input prompt, which the model analyzes internally and generates an appropriate response.

[0711] Natural language processing (NLP) technology is used for context analysis.

[0712] Output: You get the generated response text.

[0713] Specific behavior:

[0714] The server receives the text "What's the weather forecast this week?" and analyzes it to generate a response saying "There will be many sunny days this week, with a maximum temperature of 25 degrees."

[0715] Step 4: Text-to-speech response

[0716] ------

[0717] The terminal receives the response text from the server, converts it into voice, and conveys it to the user.

[0718] Input: Response text data sent from the server.

[0719] process:

[0720] The device uses speech synthesis technology (e.g., Amazon Polly) to convert the received text data into speech.

[0721] Output: Audio data is generated and played through the speaker.

[0722] Specific behavior:

[0723] The device converts the response "There will be many sunny days this week, with a maximum temperature of 25 degrees" into speech and conveys it to the user.

[0724] Step 5: Gather feedback

[0725] ------

[0726] The server stores user feedback in a database to help improve the quality of the model.

[0727] Input: Feedback data provided by the user.

[0728] process:

[0729] The server receives the feedback and stores it in a database.

[0730] The collected feedback is used to retrain or tune the generative AI model.

[0731] Output: The saved feedback data.

[0732] Specific behavior:

[0733] The user provides feedback such as "Thanks, now I can go for a walk," and the server stores this in a database, which is then used to improve the model.

[0734] This is the specific program processing flow of this system. This allows elderly people to enjoy conversations with others on a daily basis, promoting brain activity and helping to prevent dementia.

[0735] (Application example 1)

[0736] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0737] In modern society, cognitive decline among the elderly is a serious problem. In particular, lack of communication inhibits brain activation and contributes to the accelerated progression of dementia. However, many elderly people have limited opportunities to enjoy conversation with others on a daily basis, and effective methods for providing such opportunities are needed. Even community facilities and cafes for the elderly lack systems that provide consistent, high-quality conversation partners. Technology is needed to solve this problem and improve the quality of life for the elderly.

[0738] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0739] In this invention, the server includes means for disabling the generative model and initializing a model for user-specific conversation generation based on organized data including each user's conversation history, means for converting speech input from the user into text and sending the text to the server, means for generating an appropriate response using the generative model based on the text received by the server, means for using speech synthesis technology to output the generated response to the user, means for collecting feedback from the user and storing the feedback in a database for use in improving future conversation generation, and means for providing natural conversation with users at a community facility for the elderly using a smart device linked to the generative model. This allows elderly people to enjoy daily conversations even in community facilities and cafes, thereby activating their brains.

[0740] A "generative model" is an algorithm that uses artificial intelligence to generate text, images, etc.

[0741] "Conversation history for each user" refers to a record of conversations that each user has had in the past.

[0742] A "means for initializing a model" is a method for starting the operation of a generative model based on specific conditions or data.

[0743] "Means for converting voice input to text" refers to technology that converts the voice spoken by a user into text information.

[0744] "Means for transmitting to the server" refers to a method for transmitting data from the terminal to the server.

[0745] The "means for generating an appropriate response" is a technology for generating a response for forming a natural conversation based on a user's input.

[0746] "Means using speech synthesis technology" refers to technology for converting text data into speech and outputting it.

[0747] "Feedback collection methods" are methods for collecting user-provided ratings and opinions.

[0748] "Means for storing data in a database" refers to the technology for recording collected data so that it can be used later.

[0749] "Community facilities" are social institutions used by the elderly to enjoy social interactions and activities.

[0750] "Smart terminal" is a general term for highly functional portable devices, including smartphones and tablets.

[0751] "Natural conversation" refers to interaction with artificial intelligence that is similar to a human conversation.

[0752] "Brain activation" refers to stimulation and activities that enhance brain function.

[0753] This invention is a conversation support system for elderly community facilities that utilizes generative models. The aim is to provide an environment where elderly people can enjoy daily conversations and promote brain activation. The main components and operation are detailed below.

[0754] Server Processing

[0755] Initializing the AI ​​model

[0756] The server first loads the generative AI model, reads each user's conversation history, and initializes the model based on that. This enables personalized responses that reflect the user's past conversation data. An example of a generative AI model used is OpenAI's GPT-3.

[0757] Receiving and analyzing user requests

[0758] When a user sends voice input from their device, the server receives it and performs context analysis on the converted text. This allows the server to understand the user's intent and requests. The specific software used for the voice recognition technology is the Google Speech-to-Text API.

[0759] Generating responses using generative models

[0760] Based on the analysis results, the server uses a generative AI model to generate an appropriate response, which provides the best answer to the user's question.

[0761] Gathering feedback and improving the model

[0762] After the conversation is over, the server receives feedback from the user and stores it in a database, which helps improve the quality of the generative AI model.

[0763] Terminal handling

[0764] Voice to text conversion

[0765] The device receives the user's voice and converts it into text. This process uses voice recognition technology. Specific hardware used is a smart speaker (e.g., Amazon Echo) or a tablet (e.g., iPad).

[0766] Sending a text

[0767] The terminal sends the converted text to the server using a communication protocol such as HTTPS.

[0768] Receiving a response from the server and outputting audio

[0769] The device receives the response text from the server, converts it internally into speech using speech synthesis technology, and uses the Google Text-to-Speech API to play the converted speech for the user.

[0770] User Roles

[0771] Start a conversation

[0772] The user starts a conversation by speaking to the device, asking specific questions or making requests. For example, "What's the weather like today?" or "Tell me the latest news."

[0773] Response acceptance and feedback

[0774] The user listens to the responses provided by the device and provides appropriate feedback, which is used to improve the quality of future conversations.

[0775] Specific conversation examples

[0776] User: "What kind of music do you like today?"

[0777] Terminal: "Send a request to the server."

[0778] Server: Received text from user ID 123 asking, "What kind of music do you like today?" and used the generative model to generate the response, "I've been enjoying classical music lately. I think Beethoven's symphonies are particularly wonderful."

[0779] Device: "I've been enjoying classical music lately. I think Beethoven's symphonies are particularly wonderful."

[0780] User: "Thanks, that's very helpful."

[0781] This concludes the description of an embodiment of a conversation support system for elderly community facilities that uses a generative model. This system allows elderly people to enjoy conversations with others on a daily basis, promoting brain activation and contributing to the prevention of dementia.

[0782] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0783] Step 1:

[0784] Receiving voice input

[0785] Subject: Device

[0786] How it works: The device receives audio spoken by the user.

[0787] Input: User speech (e.g., "What kind of music do you like today?")

[0788] Data processing and calculation: The microphone installed on the device converts the voice data into a digital signal.

[0789] Output: Digital audio data

[0790] Step 2:

[0791] Converting audio data to text

[0792] Subject: Device

[0793] How it works: The device converts digital voice data into text.

[0794] Input: Digital audio data

[0795] Data processing and calculation: The device's voice recognition software (e.g., Google Speech-to-Text API) converts the voice data into text information.

[0796] Output: Text data (e.g., "What kind of music do you like today?")

[0797] Step 3:

[0798] Sending text data to the server

[0799] Subject: Device

[0800] Operation: The terminal sends the converted text data to the server.

[0801] Input: Text data

[0802] Data processing and calculation: Text data is sent to the server using the HTTPS protocol.

[0803] Output: Request data to the server

[0804] Step 4:

[0805] Contextual analysis of text data

[0806] Subject: Server

[0807] How it works: The server analyzes the received text data and extracts the user's intent and requests.

[0808] Input: Text data

[0809] Data processing and calculation: The server's context analysis engine analyzes the text data and identifies the intent and request.

[0810] Output: Context analysis results (e.g., identifying that the question is about music)

[0811] Step 5:

[0812] Generating an appropriate response

[0813] Subject: Server

[0814] How it works: The server generates a response using a generative AI model (e.g., OpenAI GPT-3) based on the contextual analysis results.

[0815] Input: Context analysis results

[0816] Data processing and calculation: The generative AI model generates prompt sentences and generates optimal responses.

[0817] Output: Response text data (e.g., "I've been enjoying classical music lately. I think Beethoven's symphonies are particularly wonderful.")

[0818] Step 6:

[0819] Terminal reception of response data

[0820] Subject: Device

[0821] Operation: The terminal receives the response text data sent from the server.

[0822] Input: Response text data

[0823] Data processing and calculation: The terminal receives the response data using the HTTPS protocol.

[0824] Output: Response text data (e.g., "I've been enjoying classical music lately. I think Beethoven's symphonies are particularly wonderful.")

[0825] Step 7:

[0826] Response text-to-speech

[0827] Subject: Device

[0828] Operation: The device converts the received response text data into speech.

[0829] Input: Response text data

[0830] Data processing and calculation: The device's speech synthesis software (e.g., Google Text-to-Speech API) converts the text data into voice data.

[0831] Output: Response audio data

[0832] Step 8:

[0833] Response voice output to the user

[0834] Subject: Device

[0835] Action: The device plays the audio data and communicates it to the user.

[0836] Input: Response audio data

[0837] Data processing and calculation: Plays audio through the device's built-in speaker.

[0838] Output: Response speech (e.g., "I've been enjoying classical music lately. I think Beethoven's symphonies are particularly wonderful.")

[0839] Step 9:

[0840] Gathering feedback

[0841] Subject: Device

[0842] Action: The device receives feedback from the user.

[0843] Input: User feedback

[0844] Data processing and calculation: The device converts voice input back into text or collects feedback via touch input, etc.

[0845] Output: Feedback text data

[0846] Step 10:

[0847] Feedback data sent to server and stored

[0848] Subject: Terminal and Server

[0849] Operation: The device sends the feedback text data to the server, which stores it in a database.

[0850] Input: Feedback text data

[0851] Data processing and calculation: The terminal sends the feedback text data to the server, and the server stores the received feedback data in a database.

[0852] Output: Stored feedback data

[0853] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0854] The embodiment of the present invention relates to a conversation service for preventing dementia that utilizes a generative model and an emotion engine, and the specific program processing and explanation in natural language are provided below.

[0855] An embodiment of a conversation system using a generative model and an emotion engine

[0856] The system aims to provide an environment where elderly people can enjoy daily conversations and promote the prevention of dementia through brain activation and emotion recognition. The main components and operation details are shown below.

[0857] Server Processing

[0858] 1. Initializing the AI ​​model

[0859] The server loads the generative AI model, reads each user's conversation history, and initializes the model based on that data, enabling personalized responses that reflect the user's past conversation data.

[0860] Example: The server initialized the generative AI model based on the conversation history of user ID 123.

[0861] 2. Receiving user requests and analyzing their sentiment

[0862] When a user sends a voice input from a terminal, the server receives it, converts it into text, analyzes the content, and simultaneously analyzes the user's emotions using an emotion engine to obtain emotion information.

[0863] Example: The server receives the text "How's the weather today?" and then uses an emotion engine to recognize the emotion "happy" from the user's voice.

[0864] 3. Generating responses using a generative model

[0865] Based on the analysis results and emotional information, the server uses a generative AI model to generate an appropriate response that reflects the user's current emotions.

[0866] Example: The server used a generative AI model to generate the response, "Today's weather is sunny and the temperature is 20 degrees. It looks like it's going to be a great day!"

[0867] 4. Gather feedback and improve the model

[0868] After the conversation is over, the server receives user feedback and emotion data and stores it in a database. The collected data will be used to improve the quality of the generative AI model and emotion engine.

[0869] Example: The server stores the feedback "the response was helpful" and the emotion data "fun" in a database.

[0870] Terminal handling

[0871] 1. Voice to Text

[0872] The device receives the user's voice and converts it into text data using voice recognition technology. If the voice input is "Tell me the weather," the text will also be "Tell me the weather."

[0873] Example: A user says "What's the weather?" which is converted by the device to "What's the weather?"

[0874] 2. Sending a text

[0875] The terminal sends the converted text to the server, which also includes identification information such as the user ID.

[0876] Example: The device sends the text "Tell me the weather" to the server.

[0877] 3. Receiving a response from the server and outputting audio

[0878] The device receives the response text from the server and converts it internally into speech using speech synthesis technology, which is then played back to the user.

[0879] Example: The device spoke the response to the user: "Today's weather is sunny and the temperature is 20 degrees. It looks like it's going to be a great day!"

[0880] User Roles

[0881] 1. Start a conversation

[0882] The user starts a conversation by speaking to the terminal, asking specific questions or making requests verbally.

[0883] Example: A user says, "Alexa, what's the weather like today?"

[0884] 2. Response acceptance and feedback

[0885] The user listens to the responses provided by the device and provides appropriate feedback, which is used to improve the quality of future conversations.

[0886] Example: The user provides feedback saying, "Thanks, now I can go for a walk."

[0887] Specific conversation examples

[0888] User: "Alexa, what's the weather forecast for this week?"

[0889] Terminal: "Send a request to the server."

[0890] Server: Received text from user ID 456 asking "What's the weather forecast this week?" and used the emotion engine to recognize "expectation" from the user's voice. Using the generative model, generated the response: "There will be many sunny days this week, with a maximum temperature of 25 degrees. Looking forward to it!"

[0891] Device: "There will be lots of sunny days this week, with a high of 25 degrees. Looking forward to it!"

[0892] User: "Thanks, that's very helpful."

[0893] This concludes the description of an embodiment of a conversation system using a generative model and an emotion engine. This system allows elderly people to enjoy conversations with others on a daily basis, and contributes to the prevention of dementia through brain activation and emotion recognition.

[0894] The processing flow will be explained below.

[0895] Step 1:

[0896] The server initializes the generative model. Specifically, it reads the conversation history of each user, customizes the generative model based on that data, and then initializes it.

[0897] Step 2:

[0898] The user verbally inputs questions and requests into the device in a natural conversational style.

[0899] Step 3:

[0900] The device receives the user's voice and converts it into text data using voice recognition technology. For example, if the voice input is "Tell me the weather," the text will also be "Tell me the weather."

[0901] Step 4:

[0902] The terminal sends the converted text to the server, which also includes identification information such as the user ID.

[0903] Step 5:

[0904] The server receives the text data sent from the device, analyzes the received data, and performs context analysis to understand the user's intent.

[0905] Step 6:

[0906] The server uses an emotion engine to extract the user's emotion from the speech along with context analysis. The emotion engine analyzes the tone and rate of the speech to identify the user's emotion.

[0907] Step 7:

[0908] The server uses a generative model to generate an appropriate response based on the analysis and emotional information, for example, taking into account a question about the weather and the emotion "fun" to tailor the response.

[0909] Step 8:

[0910] The server sends the generated response back to the terminal in text format. The response content is included in the sent data.

[0911] Step 9:

[0912] The device uses speech synthesis technology to generate voice data based on the received response text and outputs it to the user. For example, it might output, "Today's weather is sunny, and the temperature is 20 degrees. It looks like it's going to be a nice day today!"

[0913] Step 10:

[0914] The user receives a voice response from the terminal and provides feedback as needed, such as "Thank you, that was helpful."

[0915] Step 11:

[0916] The terminal transmits the feedback from the user to the server, and the transmitted data includes the content of the user's feedback.

[0917] Step 12:

[0918] The server analyzes the received feedback and emotion data and stores it in a database for future use in improving the generative model and emotion engine.

[0919] Example 2

[0920] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0921] Brain activation and emotional recognition through everyday conversations are important for preventing dementia in the elderly. However, it is difficult for elderly people to find suitable conversation partners on a daily basis, and conventional conversation systems are insufficient in providing individualized responses that reflect the user's emotions and past conversation history, limiting their effectiveness in preventing dementia. The present invention solves these problems and provides technology that promotes dementia prevention while allowing elderly people to enjoy everyday conversations.

[0922] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for loading a generative AI model and initializing a user-specific model based on data including each user's conversation history; means for converting voice input from the user into text and sending the text to the server; means for analyzing the text received by the server and acquiring emotional information using an emotion engine; means for generating a response based on the analysis result and emotional information using the generative AI model; means for outputting the generated response to the user by voice using speech synthesis technology; and means for collecting feedback from the user and storing the feedback and emotional data in a database. This allows elderly people to enjoy individually tailored conversations on a daily basis, and promotes dementia prevention through brain activation and emotion recognition.

[0923] A "generative AI model" is an artificial intelligence-based model used to generate conversations with users, utilizing natural language processing techniques.

[0924] "Voice input" refers to the voice that the user utters into the terminal, and is used to convert this into text data and send it to the server.

[0925] "Text-to-text" refers to the process of converting voice input into text data, which is done using speech recognition technology.

[0926] "Emotion engine" refers to software or functionality for analyzing and acquiring emotional information from the user's speech and voice.

[0927] "Response generation" refers to the process of using a generative AI model to create an appropriate response to the user based on the acquired text data and emotional information.

[0928] "Speech synthesis technology" refers to technology for converting generated response text into voice data and providing it to the user aloud.

[0929] "Conversation history for each user" is a record of past conversations with a specific user, and is used to personalize the generative AI model.

[0930] "Feedback" refers to opinions and thoughts provided by users regarding a conversation, and is used to improve the system.

[0931] "Database" refers to an information storage system for saving and managing conversation history, feedback, emotional data, etc. with users.

[0932] MODE FOR CARRYING OUT THE INVENTION

[0933] The embodiment of this invention relates to a conversation service for dementia prevention that utilizes a generative model and an emotion engine, and aims to provide an environment where elderly people can enjoy conversation on a daily basis, and promote dementia prevention through brain activation and emotion recognition. Specific program processing and details of the embodiment are described below.

[0934] Server Processing

[0935] The server first loads a generative AI model (e.g., GPT-3). Next, it reads each user's conversation history from a database and initializes the generative AI model based on that data. This initialization enables personalized responses that reflect the user's past conversation data.

[0936] When a user sends voice input from a device, the server converts the voice data into text and receives it. At the same time, it uses an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the user's emotions and obtain emotional information. Based on the analysis results and emotional information, the server uses a generative AI model to generate an appropriate response and sends the response in text format to the device.

[0937] After the conversation is over, the server stores the user's feedback and emotional data in a database, which can later be used to improve the generative AI model and emotion engine.

[0938] Terminal handling

[0939] The device receives the user's voice input and converts it into text data using voice recognition technology (e.g., Google Cloud Speech-to-Text). The converted text data is then sent from the device to a server. This text data also includes identification information such as the user ID.

[0940] When the device receives the response text from the server, it uses speech synthesis technology (e.g., Amazon Polly) to convert the received text into speech and provides it to the user.

[0941] User Roles

[0942] The user initiates a conversation by speaking to the device. At this time, they can verbally express specific questions or requests. They listen to the voice responses provided by the device and provide feedback as needed. This feedback is used to improve the quality of future conversations.

[0943] Specific conversation examples

[0944] User: "Assistant, what's the weather forecast for this week?"

[0945] Terminal: "Send a request to the server."

[0946] Server: Received text from user ID 456 asking "What's the weather forecast this week?" and used the emotion engine to recognize "expectation" from the user's voice. Using the generative AI model, generated the response: "There will be many sunny days this week, with a maximum temperature of 25 degrees. Looking forward to it!"

[0947] Device: "There will be lots of sunny days this week, with a high of 25 degrees. Looking forward to it!"

[0948] User: "Thanks, that's very helpful."

[0949] This concludes the description of an embodiment of a conversation system using a generative model and an emotion engine. This system allows elderly people to enjoy conversations with others on a daily basis, and contributes to the prevention of dementia through brain activation and emotion recognition.

[0950] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0951] Program processing flow

[0952] Server Processing

[0953] Step 1: Initializing the generative AI model

[0954] How it works: The server first loads a generative AI model (e.g., GPT-3) that is used to reflect the user's unique past conversation history.

[0955] Input: Conversation history data for each user retrieved from the database.

[0956] Data processing and calculation: The server analyzes conversation history data and applies it to a generative AI model to customize it for each user.

[0957] Output: An initialized generative AI model.

[0958] Step 2: Receiving voice input and converting it to text

[0959] What happens: The device receives the user's voice input and converts it into text using speech recognition technology (e.g., Google Cloud Speech-to-Text).

[0960] Input: User's voice input.

[0961] Data processing and data calculation: Speech to text conversion using speech recognition technology.

[0962] Output: Text data (e.g. "Tell me the weather").

[0963] Step 3: Send a text

[0964] Specific operation: The terminal sends the converted text data along with the user ID to the server.

[0965] Input: Text data and user ID.

[0966] Data processing and data calculation: Formation of data structures for sending text data and identification information to the server.

[0967] Output: The text data sent to the server and the user ID.

[0968] Step 4: Text and sentiment analysis

[0969] Specific operation: The server analyzes the received text data and obtains the user's emotional information using an emotion engine (e.g., IBM Watson Tone Analyzer).

[0970] Input: Received text data.

[0971] Data processing and data calculation: Sentiment analysis and text content analysis using emotion engines.

[0972] Output: Analysis results and user sentiment information.

[0973] Step 5: Generate a response

[0974] How it works: The server uses a generative AI model to generate an appropriate response based on the analysis results and emotional information.

[0975] Input: Analysis results and emotion information.

[0976] Data processing and data calculation: Response generation using generative AI models.

[0977] Output: Response text (e.g. "Today the weather is sunny and the temperature is 20 degrees. It looks like it's going to be a great day!").

[0978] Step 6: Sending a Response

[0979] Specific operation: The server sends the generated response text to the terminal.

[0980] Input: The response text.

[0981] Data processing and data calculation: Transfer of response text.

[0982] Output: Sending the response text from the server to the terminal.

[0983] Step 7: Transcribing and outputting responses

[0984] Specific operation: The device converts the received response text into speech using speech synthesis technology (e.g., Amazon Polly) and provides it to the user.

[0985] Input: The response text.

[0986] Data processing and data arithmetic: Response text to speech conversion.

[0987] Output: A spoken response (e.g. "Today the weather is sunny and the temperature is 20 degrees. It's going to be a great day!").

[0988] Step 8: Collect and store feedback

[0989] Specific operation: After the conversation ends, the server stores the user's feedback and emotion data in a database.

[0990] Input: User feedback and sentiment data.

[0991] Data processing and data computation: Analysis and storage of feedback and emotion data.

[0992] Output: Feedback and sentiment data stored in a database.

[0993] User Action

[0994] Step 1: Start a conversation

[0995] Specific actions: The user speaks to the device and verbally communicates a specific question or request.

[0996] Input: User's voice.

[0997] Data processing and data arithmetic: Shaping what we say.

[0998] Output: Voice input to the device (e.g., "Assistant, what's the weather today?").

[0999] Step 2: Acceptance of responses and feedback

[1000] Specific operation: The user listens to the voice response from the device and provides appropriate feedback based on the content.

[1001] Input: A voice response from the device.

[1002] Data processing and data arithmetic: Shaping feedback based on response content.

[1003] Output: User feedback (e.g. "Thanks, that's very helpful").

[1004] These are the specific processing steps of this system's program. This allows elderly people to enjoy conversations with others on a daily basis, and contributes to the prevention of dementia through brain activation and emotional recognition.

[1005] (Application example 2)

[1006] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1007] To enable elderly people to use food delivery services smoothly, simple voice-activated operation is required. However, conventional systems have difficulty responding to individual emotions, resulting in a poor user experience. Furthermore, they are unable to generate personalized suggestions or responses based on order history, which may lead to a decrease in user satisfaction. There is a need to provide a system that can solve these issues and enable elderly people to use food delivery services comfortably.

[1008] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1009] In this invention, the server includes means for disabling the generative model and initializing a user-specific model for conversation generation based on organized data including each user's conversation history, means for converting speech input from the user into text and sending the text to the server, means for generating appropriate responses using the generative model based on the text received by the server, means for using speech synthesis technology to output the generated responses to the user, means for collecting feedback from the user and storing the feedback in a database for use in improving future conversation generation, means for using an emotion engine to ensure that suggested responses reflect the user's emotional state, and means for suggesting appropriate menu items to the user based on emotions and past ordering history. This allows for the generation of personalized responses based on the user's emotions, enabling elderly people to smoothly use food delivery services in an emotionally satisfying manner.

[1010] 1. A "generative model" is a machine learning algorithm that generates appropriate responses based on user input data.

[1011] 2. "Conversation history" is a record of past conversations a user has had, and is data used to provide personalized support in future conversations.

[1012] 3. "Voice input" refers to voice data uttered by a user through a device such as a microphone.

[1013] 4. "Convert to text" is the process of converting voice input into text data.

[1014] 5. “Server” means a computer system that hosts Generative Models and processes data received from Users.

[1015] 6. An "emotion engine" is an algorithm that analyzes a user's emotions from voice and text data and identifies their emotional state.

[1016] 7. "Speech synthesis technology" means technology that converts text data into voice data and is used to communicate generated responses to the user by voice.

[1017] 8. "Feedback" means the responses or opinions provided by a user and any emotional data associated with those responses.

[1018] 9. "Order History" means a record of past orders placed by a User and is used to make recommendations for future orders.

[1019] 10. "Menu Suggestions" are a list of foods or items that the system suggests to the user based on past ordering history and the user's emotional state.

[1020] This invention relates to a food delivery ordering support system for seniors that utilizes a generative model and an emotion engine. The system enables seniors to easily order food delivery using voice and provides personalized suggestions through emotion analysis to enhance the user experience.

[1021] System Overview

[1022] Hardware

[1023] Smartphones: Devices used by seniors that support voice input and output.

[1024] Server: A computer system that hosts the generative model and emotion engine and processes data received from users.

[1025] software

[1026] Generative AI model (GPT-2): A machine learning algorithm used to generate appropriate responses to user input.

[1027] Emotion engine: Software for analyzing user emotions from voice and text data to identify emotional states. For example, we use the EmotionAnalyzer library.

[1028] Speech recognition technology: Technology to convert the elderly's voice input into text. As an example, we will use the Google Speech-to-Text API.

[1029] Speech synthesis technology: Technology to convert responses from the server into voice data and convey them to the user. As an example, we will use the Google Text-to-Speech API.

[1030] System processing flow

[1031] Server Processing

[1032] 1. Model initialization: The server loads the generative model, reads each user's order history, and initializes the model based on that data. This allows for personalized responses that reflect the user's past order data.

[1033] 2. Receiving and analyzing voice input: The terminal receives voice input from the elderly person and analyzes the converted text. At the same time, it uses an emotion engine to analyze the user's emotions and obtain emotional information.

[1034] 3. Response and suggestion generation: Based on the analysis results and sentiment information, a generative model is used to generate appropriate responses and suggestions. For example, suggesting appropriate menu items based on the user's sentiment and past ordering history.

[1035] 4. Feedback collection: After the conversation is over, the server receives feedback and sentiment data from the user and stores it in a database, which can improve the quality of responses in the future.

[1036] Terminal handling

[1037] 1. Voice input and text conversion: The device receives the voice spoken by the elderly person and converts it into text data using voice recognition technology.

[1038] 2. Sending the text data: The converted text data is sent to the server, including identification information such as the user ID.

[1039] 3. Receiving response and outputting voice: Receives the response text from the server, converts it into voice using speech synthesis technology, and plays it back to the user.

[1040] User Roles

[1041] 1. Start the conversation: The elderly person starts speaking to the device and placing an order by voice.

[1042] 2. Response acceptance and feedback: The elderly person listens to the response from the device, confirms the order, and provides feedback. The feedback is used to improve the quality of response generation in future.

[1043] Specific examples

[1044] User: "Can you help me place an order?"

[1045] Terminal: "Of course. What would you like to order today?"

[1046] User: "I want pizza."

[1047] Server: Based on user ID 123's order history, suggest a menu item that matches their order history, saying, "How about pizza? Margherita or pepperoni are also delicious. Which would you like?"

[1048] Terminal: "How about some pizza? Margherita or pepperoni are both delicious. Which would you like?"

[1049] User: "Margherita, please."

[1050] Terminal: "Okay, I'll order a Margherita pizza!"

[1051] Prompt Sentence Examples

[1052] User input: "I want pizza."

[1053] Sentiment analysis result: “Expectation”

[1054] Suggestion generation prompt: "The user has a sense of anticipation. Suggest Margherita pizza or pepperoni pizza based on their past orders."

[1055] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1056] Step 1:

[1057] Initializing the Server

[1058] The server loads the generative model and initializes it with each user's order history, allowing it to generate user-specific responses based on past data.

[1059] Input: User ID and order history data

[1060] Processing: The server analyzes the data for each user and adjusts the generative model

[1061] Output: An initialized generative model

[1062] Step 2:

[1063] Receiving and converting voice input

[1064] The user speaks to the smartphone to input voice, which the device receives and converts into text using voice recognition technology.

[1065] Input: User voice input

[1066] Processing: Using speech recognition technology (e.g., Google Speech-to-Text API) to convert voice data into text data

[1067] Output: Text data

[1068] Step 3:

[1069] Sending text data

[1070] The device sends the converted text data to the server, which also includes identification information such as the user ID.

[1071] Input: Text data and user ID

[1072] Process: Send the text data and user ID to the server

[1073] Output: The input text received by the server

[1074] Step 4:

[1075] Emotion analysis

[1076] The server analyzes the user's emotions using an emotion engine based on the received text data.

[1077] Input: Text data

[1078] Processing: Analyze user emotions using an emotion engine (e.g., EmotionAnalyzer library)

[1079] Output: Emotional state data

[1080] Step 5:

[1081] Generate responses and suggestions

[1082] Generative AI models are used to generate appropriate responses and suggestions based on sentiment information and past order history.

[1083] Input: Text data, emotional state data, order history data

[1084] Processing: Generative AI models generate appropriate responses and menu suggestions based on the input data.

[1085] Output: Response text data

[1086] Step 6:

[1087] Audio output of the response

[1088] The terminal receives the response text data from the server, converts it into voice data using voice synthesis technology, and plays it back to the user.

[1089] Input: Response text data

[1090] Processing: Convert text data into audio data using speech synthesis technology (e.g., Google Text-to-Speech API).

[1091] Output: The audio data to be played back to the user

[1092] Step 7:

[1093] Gathering feedback

[1094] After the conversation is finished, the user provides feedback to the terminal, which sends this feedback to the server, which stores it in a database.

[1095] Input: User feedback

[1096] Processing: The server records the feedback and emotional state data and uses it to improve the next conversation generation.

[1097] Output: Data for an improved generative model

[1098] As a result of the above, the system allows seniors to easily order food delivery using voice commands and provides personalized service through sentiment analysis.

[1099] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1100] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1101] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1102] [Third embodiment]

[1103] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1104] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1105] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1106] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1107] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1108] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1109] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1110] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1111] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1112] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1113] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1114] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1115] The embodiment of the present invention relates to a conversation service for preventing dementia that utilizes a generative model, and the specific program processing and explanation in natural language are provided below.

[1116] An embodiment of a conversation system using a generative model

[1117] The purpose of this system is to provide an environment where elderly people can enjoy daily conversations and promote brain activation. The main components and operation details are shown below.

[1118] Server Processing

[1119] 1. Initializing the AI ​​model

[1120] The server first loads the generative AI model, reads each user's conversation history, and initializes the model based on that information. This enables personalized responses that reflect the user's past conversation data.

[1121] Example: The server initialized the generative AI model based on the conversation history of user ID 123.

[1122] 2. Receiving and analyzing user requests

[1123] When a user sends voice input from a device, the server receives it, converts it into text, and performs context analysis to understand the user's intent and requests.

[1124] Example: The server receives the text "How's the weather today?" and interprets it as a question about the weather.

[1125] 3. Generating responses using a generative model

[1126] Based on the analysis results, the server uses a generative AI model to generate an appropriate response that provides the best answer to the user's question.

[1127] Example: The server uses a generative AI model to generate the response, "Today's weather is sunny and the temperature is 25 degrees."

[1128] 4. Gather feedback and improve the model

[1129] After the conversation is over, the server receives feedback from the user and stores it in a database, which helps improve the quality of the generative AI model.

[1130] Example: The server stores the feedback "the response was helpful" in its database.

[1131] Terminal handling

[1132] 1. Voice to Text

[1133] The device receives the user's voice and converts it into text using voice recognition technology.

[1134] Example: A user says "What's the weather?" which is converted by the device to "What's the weather?"

[1135] 2. Sending a text

[1136] The terminal sends the converted text to the server.

[1137] Example: The device sends the text "Tell me the weather" to the server.

[1138] 3. Receiving a response from the server and outputting audio

[1139] The device receives the response text from the server and converts it internally into speech using speech synthesis technology, which is then played back to the user.

[1140] Example: The device spoke the response to the user: "Today's weather is sunny and the temperature is 25 degrees."

[1141] User Roles

[1142] 1. Start a conversation

[1143] The user starts a conversation by speaking to the device, asking specific questions or making requests.

[1144] Example: A user says, "Alexa, what's the weather like today?"

[1145] 2. Response acceptance and feedback

[1146] The user listens to the responses provided by the device and provides appropriate feedback, which is used to improve the quality of future conversations.

[1147] Example: The user provides feedback saying, "Thanks, now I can go for a walk."

[1148] Specific conversation examples

[1149] User: "Alexa, what's the weather forecast for this week?"

[1150] Terminal: "Send a request to the server."

[1151] Server: Received text from user ID 456 asking "What's the weather forecast this week?" and used the generative model to generate the response "There will be many sunny days this week, with a maximum temperature of 25 degrees."

[1152] Device: "There will be many sunny days this week, with a high of 25 degrees," the user is told.

[1153] User: "Thanks, that's very helpful."

[1154] This concludes the description of an embodiment of a conversation system using a generative model. This system allows elderly people to enjoy conversations with others on a daily basis, promoting brain activity and contributing to the prevention of dementia.

[1155] The processing flow will be explained below.

[1156] Step 1:

[1157] The server initializes the generative model. Specifically, it reads the conversation history of each user, customizes the generative model based on that data, and then initializes it.

[1158] Step 2:

[1159] The user verbally inputs questions and requests into the device in a natural conversational style.

[1160] Step 3:

[1161] The device receives the user's voice and converts it into text data using voice recognition technology. If the voice input is "Tell me the weather," the text will also be "Tell me the weather."

[1162] Step 4:

[1163] The terminal sends the converted text to the server, which also includes identification information such as the user ID.

[1164] Step 5:

[1165] The server receives the text data sent from the device, analyzes the received data, and performs context analysis to understand the user's intent.

[1166] Step 6:

[1167] The server uses the generative model to generate an appropriate response based on the analysis results, such as "Today's weather is sunny and the temperature is 20 degrees."

[1168] Step 7:

[1169] The server generates a response and sends it back to the terminal in text format, which contains the response content.

[1170] Step 8:

[1171] The terminal generates voice data based on the received response text using speech synthesis technology and outputs the voice to the user.

[1172] Step 9:

[1173] The user receives a voice response from the terminal and provides feedback as needed, such as "Thank you, that was helpful."

[1174] Step 10:

[1175] The terminal transmits the feedback from the user back to the server, and the transmitted data includes the content of the user's feedback.

[1176] Step 11:

[1177] The server analyzes the received feedback data and stores it in a database for future improvements to the generative model. This feedback data is used as reference information for future conversation generation.

[1178] Example 1

[1179] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1180] To contribute to the prevention of dementia among the elderly, it is important to provide an environment where people can enjoy everyday conversations. However, providing conversations that meet individual needs requires advanced natural language processing technology and responses that reflect the user's past conversation history. Furthermore, to improve the quality of the generated responses, a system is needed to efficiently collect user feedback and use it to improve the model. However, current technology does not provide a system that can consistently achieve these goals.

[1181] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1182] In this invention, the server includes means for loading a generative AI model and reading each user's conversation history to initialize the model, means for converting voice input from the user into text and sending the text to the server, means for the server to perform context analysis of the received text and generate an appropriate response using the generative AI model, means for using speech synthesis technology to output the generated response to the user, and means for collecting feedback from users and storing the feedback in a database for use in improving future conversation generation, thereby providing conversations tailored to individual needs and enabling continuous improvement of the model using the feedback.

[1183] A "generative AI model" is a machine learning model trained to generate text and data using artificial intelligence (AI).

[1184] "Conversation history" is a record of past conversations that a particular user has had, data that can be used to personalize future responses.

[1185] "Speech recognition" is the technology that analyzes voice data and converts it into corresponding text.

[1186] "Context analysis" is a natural language processing technique used to understand the content and intent of input text.

[1187] "Speech synthesis technology" is a technology that converts text data into speech and generates artificially synthesized speech.

[1188] "Feedback" refers to ratings and comments on responses provided by users, and is information used to improve the system.

[1189] A "database" is a collection of information that stores data in an organized manner and can be efficiently accessed when needed.

[1190] "Natural language processing technology" is technology for understanding, interpreting, and generating human language.

[1191] "Machine learning" is a technique that uses data to improve algorithms for pattern recognition and prediction.

[1192] This invention relates to a conversation system for dementia prevention using a generative AI model. In particular, it provides a mechanism for initializing the generative model based on an individual conversation history, converting speech input into text, generating responses using the generative model, synthesizing speech output, and collecting and learning feedback.

[1193] Overall system configuration

[1194] This system mainly consists of three main components: a server, a terminal, and a user. The server has the generative AI model, the terminal is responsible for speech recognition and speech synthesis, and the user has a natural conversation with the system. Each component is described in detail below.

[1195] Server Roles

[1196] Loading and initializing the generative AI model

[1197] The server loads a generative AI model (e.g., GPT-4) and runs the model using a high-performance GPU. Each user's conversation history is retrieved from a database and input into the model for initialization. This enables personalized responses that reflect the user's past conversation data.

[1198] Receiving and analyzing user requests

[1199] The server receives voice input from the device and converts it to text using a speech recognition engine (e.g., Google Speech-to-Text API). The converted text is then contextually analyzed using a generative AI model to understand the user's intent.

[1200] Response generation using generative models

[1201] Based on the analysis results, the server uses a generative AI model to generate an appropriate response to the user's question or request. In this process, the user's question is given as an input prompt, and the model generates an appropriate answer.

[1202] Example: In response to a user question, "What's the weather forecast this week?", the generative AI model generates the response, "There will be many sunny days this week, with a maximum temperature of 25 degrees."

[1203] Gathering feedback and improving the model

[1204] After the conversation, the server receives feedback from the user and stores it in a database. The collected feedback is used to improve the quality of the generative AI model and for continuous learning.

[1205] Device Role

[1206] Voice to text conversion

[1207] The device collects the user's voice using a built-in microphone and converts it into text using voice recognition technology (e.g., Google Speech-to-Text API). This conversion process ensures that the system accurately conveys natural speech from the user.

[1208] Sending text and voice output

[1209] The device sends the converted text to the server and receives a response from the server. The received response text is converted into speech using speech synthesis technology (e.g., Amazon Polly) and conveyed to the user.

[1210] Example: The device will speak the response to the user, "There will be many sunny days this week, with a maximum temperature of 25 degrees."

[1211] User Roles

[1212] Initiating a conversation and accepting a response

[1213] The user initiates a conversation by speaking into the device, asking specific questions or making requests, listening to the responses provided by the device and providing feedback as needed.

[1214] For example, when a user says, "Assistant, what's the weather like today?", the system responds by providing an appropriate response.

[1215] Specific examples

[1216] 1. User: "Assistant, what's the weather forecast for this week?"

[1217] 2. Device: Converts speech to text and sends it to the server.

[1218] 3. Server: Input the text "What's the weather forecast this week?" into the generative AI model and generate a response.

[1219] 4. Server: Sends the response "There will be many sunny days this week, and the maximum temperature will be 25 degrees" to the terminal.

[1220] 5. Terminal: Converts the received text into speech and conveys it to the user.

[1221] 6. User: "Thanks, that would be great."

[1222] In this way, elderly people can enjoy conversation with someone on a daily basis, which can promote brain activation and contribute to the prevention of dementia.

[1223] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1224] Step 1: Load and initialize the AI ​​model

[1225] ------

[1226] The server loads the generative AI model and reads the conversation history for each user.

[1227] Input: The server retrieves a file from the database containing the user ID, the corresponding conversation history, and the generative AI model (e.g., GPT-4).

[1228] process:

[1229] The server loads the generative AI model using a high-performance GPU.

[1230] Next, we load the past conversation history from the database based on the user ID and use this history to initialize the model.

[1231] Output: The initialized generative AI model runs in memory and reflects the conversational context of each user.

[1232] Specific behavior:

[1233] The server loads the GPT-4 model into memory on a GPU server in the data center.

[1234] The server retrieves the past conversation log of user ID 123 from the database and incorporates it into the model.

[1235] Step 2: Convert voice to text

[1236] ------

[1237] The terminal converts the user's voice into text and sends it to the server.

[1238] Input: Voice data spoken by the user into the device.

[1239] process:

[1240] The device's built-in microphone collects the user's voice and converts it into text using a local or cloud-based speech recognition engine (e.g., Google Speech-to-Text API).

[1241] Output: Text data is generated and sent from the terminal to the server.

[1242] Specific behavior:

[1243] When a user says, "Alexa, what's the weather today?", the device records this speech and converts it into text: "Alexa, what's the weather today?"

[1244] Step 3: Contextual analysis of the text

[1245] ------

[1246] The server parses the received text and uses a generative AI model to generate an appropriate response.

[1247] Input: User's text data received by the server from the device.

[1248] process:

[1249] The server provides the generative AI model with the user's text as an input prompt, which the model analyzes internally and generates an appropriate response.

[1250] Natural language processing (NLP) technology is used for context analysis.

[1251] Output: You get the generated response text.

[1252] Specific behavior:

[1253] The server receives the text "What's the weather forecast this week?" and analyzes it to generate a response saying "There will be many sunny days this week, with a maximum temperature of 25 degrees."

[1254] Step 4: Text-to-speech response

[1255] ------

[1256] The terminal receives the response text from the server, converts it into voice, and conveys it to the user.

[1257] Input: Response text data sent from the server.

[1258] process:

[1259] The device uses speech synthesis technology (e.g., Amazon Polly) to convert the received text data into speech.

[1260] Output: Audio data is generated and played through the speaker.

[1261] Specific behavior:

[1262] The device converts the response "There will be many sunny days this week, with a maximum temperature of 25 degrees" into speech and conveys it to the user.

[1263] Step 5: Gather feedback

[1264] ------

[1265] The server stores user feedback in a database to help improve the quality of the model.

[1266] Input: Feedback data provided by the user.

[1267] process:

[1268] The server receives the feedback and stores it in a database.

[1269] The collected feedback is used to retrain or tune the generative AI model.

[1270] Output: The saved feedback data.

[1271] Specific behavior:

[1272] The user provides feedback such as "Thanks, now I can go for a walk," and the server stores this in a database, which is then used to improve the model.

[1273] This is the specific program processing flow of this system. This allows elderly people to enjoy conversations with others on a daily basis, promoting brain activity and helping to prevent dementia.

[1274] (Application example 1)

[1275] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1276] In modern society, cognitive decline among the elderly is a serious problem. In particular, lack of communication inhibits brain activation and contributes to the accelerated progression of dementia. However, many elderly people have limited opportunities to enjoy conversation with others on a daily basis, and effective methods for providing such opportunities are needed. Even community facilities and cafes for the elderly lack systems that provide consistent, high-quality conversation partners. Technology is needed to solve this problem and improve the quality of life for the elderly.

[1277] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1278] In this invention, the server includes means for disabling the generative model and initializing a model for user-specific conversation generation based on organized data including each user's conversation history, means for converting speech input from the user into text and sending the text to the server, means for generating an appropriate response using the generative model based on the text received by the server, means for using speech synthesis technology to output the generated response to the user, means for collecting feedback from the user and storing the feedback in a database for use in improving future conversation generation, and means for providing natural conversation with users at a community facility for the elderly using a smart device linked to the generative model. This allows elderly people to enjoy daily conversations even in community facilities and cafes, thereby activating their brains.

[1279] A "generative model" is an algorithm that uses artificial intelligence to generate text, images, etc.

[1280] "Conversation history for each user" refers to a record of conversations that each user has had in the past.

[1281] A "means for initializing a model" is a method for starting the operation of a generative model based on specific conditions or data.

[1282] "Means for converting voice input to text" refers to technology that converts the voice spoken by a user into text information.

[1283] "Means for transmitting to the server" refers to a method for transmitting data from the terminal to the server.

[1284] The "means for generating an appropriate response" is a technology for generating a response for forming a natural conversation based on a user's input.

[1285] "Means using speech synthesis technology" refers to technology for converting text data into speech and outputting it.

[1286] "Feedback collection methods" are methods for collecting user-provided ratings and opinions.

[1287] "Means for storing data in a database" refers to the technology for recording collected data so that it can be used later.

[1288] "Community facilities" are social institutions used by the elderly to enjoy social interactions and activities.

[1289] "Smart terminal" is a general term for highly functional portable devices, including smartphones and tablets.

[1290] "Natural conversation" refers to interaction with artificial intelligence that is similar to a human conversation.

[1291] "Brain activation" refers to stimulation and activities that enhance brain function.

[1292] This invention is a conversation support system for elderly community facilities that utilizes generative models. The aim is to provide an environment where elderly people can enjoy daily conversations and promote brain activation. The main components and operation are detailed below.

[1293] Server Processing

[1294] Initializing the AI ​​model

[1295] The server first loads the generative AI model, reads each user's conversation history, and initializes the model based on that. This enables personalized responses that reflect the user's past conversation data. An example of a generative AI model used is OpenAI's GPT-3.

[1296] Receiving and analyzing user requests

[1297] When a user sends voice input from their device, the server receives it and performs context analysis on the converted text. This allows the server to understand the user's intent and requests. The specific software used for the voice recognition technology is the Google Speech-to-Text API.

[1298] Generating responses using generative models

[1299] Based on the analysis results, the server uses a generative AI model to generate an appropriate response, which provides the best answer to the user's question.

[1300] Gathering feedback and improving the model

[1301] After the conversation is over, the server receives feedback from the user and stores it in a database, which helps improve the quality of the generative AI model.

[1302] Terminal handling

[1303] Voice to text conversion

[1304] The device receives the user's voice and converts it into text. This process uses voice recognition technology. Specific hardware used is a smart speaker (e.g., Amazon Echo) or a tablet (e.g., iPad).

[1305] Sending a text

[1306] The terminal sends the converted text to the server using a communication protocol such as HTTPS.

[1307] Receiving a response from the server and outputting audio

[1308] The device receives the response text from the server, converts it internally into speech using speech synthesis technology, and uses the Google Text-to-Speech API to play the converted speech for the user.

[1309] User Roles

[1310] Start a conversation

[1311] The user starts a conversation by speaking to the device, asking specific questions or making requests. For example, "What's the weather like today?" or "Tell me the latest news."

[1312] Response acceptance and feedback

[1313] The user listens to the responses provided by the device and provides appropriate feedback, which is used to improve the quality of future conversations.

[1314] Specific conversation examples

[1315] User: "What kind of music do you like today?"

[1316] Terminal: "Send a request to the server."

[1317] Server: Received text from user ID 123 asking, "What kind of music do you like today?" and used the generative model to generate the response, "I've been enjoying classical music lately. I think Beethoven's symphonies are particularly wonderful."

[1318] Device: "I've been enjoying classical music lately. I think Beethoven's symphonies are particularly wonderful."

[1319] User: "Thanks, that's very helpful."

[1320] This concludes the description of an embodiment of a conversation support system for elderly community facilities that uses a generative model. This system allows elderly people to enjoy conversations with others on a daily basis, promoting brain activation and contributing to the prevention of dementia.

[1321] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1322] Step 1:

[1323] Receiving voice input

[1324] Subject: Device

[1325] How it works: The device receives audio spoken by the user.

[1326] Input: User speech (e.g., "What kind of music do you like today?")

[1327] Data processing and calculation: The microphone installed on the device converts the voice data into a digital signal.

[1328] Output: Digital audio data

[1329] Step 2:

[1330] Converting audio data to text

[1331] Subject: Device

[1332] How it works: The device converts digital voice data into text.

[1333] Input: Digital audio data

[1334] Data processing and calculation: The device's voice recognition software (e.g., Google Speech-to-Text API) converts the voice data into text information.

[1335] Output: Text data (e.g., "What kind of music do you like today?")

[1336] Step 3:

[1337] Sending text data to the server

[1338] Subject: Device

[1339] Operation: The terminal sends the converted text data to the server.

[1340] Input: Text data

[1341] Data processing and calculation: Text data is sent to the server using the HTTPS protocol.

[1342] Output: Request data to the server

[1343] Step 4:

[1344] Contextual analysis of text data

[1345] Subject: Server

[1346] How it works: The server analyzes the received text data and extracts the user's intent and requests.

[1347] Input: Text data

[1348] Data processing and calculation: The server's context analysis engine analyzes the text data and identifies the intent and request.

[1349] Output: Context analysis results (e.g., identifying that the question is about music)

[1350] Step 5:

[1351] Generating an appropriate response

[1352] Subject: Server

[1353] How it works: The server generates a response using a generative AI model (e.g., OpenAI GPT-3) based on the contextual analysis results.

[1354] Input: Context analysis results

[1355] Data processing and calculation: The generative AI model generates prompt sentences and generates optimal responses.

[1356] Output: Response text data (e.g., "I've been enjoying classical music lately. I think Beethoven's symphonies are particularly wonderful.")

[1357] Step 6:

[1358] Terminal reception of response data

[1359] Subject: Device

[1360] Operation: The terminal receives the response text data sent from the server.

[1361] Input: Response text data

[1362] Data processing and calculation: The terminal receives the response data using the HTTPS protocol.

[1363] Output: Response text data (e.g., "I've been enjoying classical music lately. I think Beethoven's symphonies are particularly wonderful.")

[1364] Step 7:

[1365] Response text-to-speech

[1366] Subject: Device

[1367] Operation: The device converts the received response text data into speech.

[1368] Input: Response text data

[1369] Data processing and calculation: The device's speech synthesis software (e.g., Google Text-to-Speech API) converts the text data into voice data.

[1370] Output: Response audio data

[1371] Step 8:

[1372] Response voice output to the user

[1373] Subject: Device

[1374] Action: The device plays the audio data and communicates it to the user.

[1375] Input: Response audio data

[1376] Data processing and calculation: Plays audio through the device's built-in speaker.

[1377] Output: Response speech (e.g., "I've been enjoying classical music lately. I think Beethoven's symphonies are particularly wonderful.")

[1378] Step 9:

[1379] Gathering feedback

[1380] Subject: Device

[1381] Action: The device receives feedback from the user.

[1382] Input: User feedback

[1383] Data processing and calculation: The device converts voice input back into text or collects feedback via touch input, etc.

[1384] Output: Feedback text data

[1385] Step 10:

[1386] Feedback data sent to server and stored

[1387] Subject: Terminal and Server

[1388] Operation: The device sends the feedback text data to the server, which stores it in a database.

[1389] Input: Feedback text data

[1390] Data processing and calculation: The terminal sends the feedback text data to the server, and the server stores the received feedback data in a database.

[1391] Output: Stored feedback data

[1392] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1393] The embodiment of the present invention relates to a conversation service for preventing dementia that utilizes a generative model and an emotion engine, and the specific program processing and explanation in natural language are provided below.

[1394] An embodiment of a conversation system using a generative model and an emotion engine

[1395] The system aims to provide an environment where elderly people can enjoy daily conversations and promote the prevention of dementia through brain activation and emotion recognition. The main components and operation details are shown below.

[1396] Server Processing

[1397] 1. Initializing the AI ​​model

[1398] The server loads the generative AI model, reads each user's conversation history, and initializes the model based on that data, enabling personalized responses that reflect the user's past conversation data.

[1399] Example: The server initialized the generative AI model based on the conversation history of user ID 123.

[1400] 2. Receiving user requests and analyzing their sentiment

[1401] When a user sends a voice input from a terminal, the server receives it, converts it into text, analyzes the content, and simultaneously analyzes the user's emotions using an emotion engine to obtain emotion information.

[1402] Example: The server receives the text "How's the weather today?" and then uses an emotion engine to recognize the emotion "happy" from the user's voice.

[1403] 3. Generating responses using a generative model

[1404] Based on the analysis results and emotional information, the server uses a generative AI model to generate an appropriate response that reflects the user's current emotions.

[1405] Example: The server used a generative AI model to generate the response, "Today's weather is sunny and the temperature is 20 degrees. It looks like it's going to be a great day!"

[1406] 4. Gather feedback and improve the model

[1407] After the conversation is over, the server receives user feedback and emotion data and stores it in a database. The collected data will be used to improve the quality of the generative AI model and emotion engine.

[1408] Example: The server stores the feedback "the response was helpful" and the emotion data "fun" in a database.

[1409] Terminal handling

[1410] 1. Voice to Text

[1411] The device receives the user's voice and converts it into text data using voice recognition technology. If the voice input is "Tell me the weather," the text will also be "Tell me the weather."

[1412] Example: A user says "What's the weather?" which is converted by the device to "What's the weather?"

[1413] 2. Sending a text

[1414] The terminal sends the converted text to the server, which also includes identification information such as the user ID.

[1415] Example: The device sends the text "Tell me the weather" to the server.

[1416] 3. Receiving a response from the server and outputting audio

[1417] The device receives the response text from the server and converts it internally into speech using speech synthesis technology, which is then played back to the user.

[1418] Example: The device spoke the response to the user: "Today's weather is sunny and the temperature is 20 degrees. It looks like it's going to be a great day!"

[1419] User Roles

[1420] 1. Start a conversation

[1421] The user starts a conversation by speaking to the terminal, asking specific questions or making requests verbally.

[1422] Example: A user says, "Alexa, what's the weather like today?"

[1423] 2. Response acceptance and feedback

[1424] The user listens to the responses provided by the device and provides appropriate feedback, which is used to improve the quality of future conversations.

[1425] Example: The user provides feedback saying, "Thanks, now I can go for a walk."

[1426] Specific conversation examples

[1427] User: "Alexa, what's the weather forecast for this week?"

[1428] Terminal: "Send a request to the server."

[1429] Server: Received text from user ID 456 asking "What's the weather forecast this week?" and used the emotion engine to recognize "expectation" from the user's voice. Using the generative model, generated the response: "There will be many sunny days this week, with a maximum temperature of 25 degrees. Looking forward to it!"

[1430] Device: "There will be lots of sunny days this week, with a high of 25 degrees. Looking forward to it!"

[1431] User: "Thanks, that's very helpful."

[1432] This concludes the description of an embodiment of a conversation system using a generative model and an emotion engine. This system allows elderly people to enjoy conversations with others on a daily basis, and contributes to the prevention of dementia through brain activation and emotion recognition.

[1433] The processing flow will be explained below.

[1434] Step 1:

[1435] The server initializes the generative model. Specifically, it reads the conversation history of each user, customizes the generative model based on that data, and then initializes it.

[1436] Step 2:

[1437] The user verbally inputs questions and requests into the device in a natural conversational style.

[1438] Step 3:

[1439] The device receives the user's voice and converts it into text data using voice recognition technology. For example, if the voice input is "Tell me the weather," the text will also be "Tell me the weather."

[1440] Step 4:

[1441] The terminal sends the converted text to the server, which also includes identification information such as the user ID.

[1442] Step 5:

[1443] The server receives the text data sent from the device, analyzes the received data, and performs context analysis to understand the user's intent.

[1444] Step 6:

[1445] The server uses an emotion engine to extract the user's emotion from the speech along with context analysis. The emotion engine analyzes the tone and rate of the speech to identify the user's emotion.

[1446] Step 7:

[1447] The server uses a generative model to generate an appropriate response based on the analysis and emotional information, for example, taking into account a question about the weather and the emotion "fun" to tailor the response.

[1448] Step 8:

[1449] The server sends the generated response back to the terminal in text format. The response content is included in the sent data.

[1450] Step 9:

[1451] The device uses speech synthesis technology to generate voice data based on the received response text and outputs it to the user. For example, it might output, "Today's weather is sunny, and the temperature is 20 degrees. It looks like it's going to be a nice day today!"

[1452] Step 10:

[1453] The user receives a voice response from the terminal and provides feedback as needed, such as "Thank you, that was helpful."

[1454] Step 11:

[1455] The terminal transmits the feedback from the user to the server, and the transmitted data includes the content of the user's feedback.

[1456] Step 12:

[1457] The server analyzes the received feedback and emotion data and stores it in a database for future use in improving the generative model and emotion engine.

[1458] Example 2

[1459] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1460] Brain activation and emotional recognition through everyday conversations are important for preventing dementia in the elderly. However, it is difficult for elderly people to find suitable conversation partners on a daily basis, and conventional conversation systems are insufficient in providing individualized responses that reflect the user's emotions and past conversation history, limiting their effectiveness in preventing dementia. The present invention solves these problems and provides technology that promotes dementia prevention while allowing elderly people to enjoy everyday conversations.

[1461] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for loading a generative AI model and initializing a user-specific model based on data including each user's conversation history; means for converting voice input from the user into text and sending the text to the server; means for analyzing the text received by the server and acquiring emotional information using an emotion engine; means for generating a response based on the analysis result and emotional information using the generative AI model; means for outputting the generated response to the user by voice using speech synthesis technology; and means for collecting feedback from the user and storing the feedback and emotional data in a database. This allows elderly people to enjoy individually tailored conversations on a daily basis, and promotes dementia prevention through brain activation and emotion recognition.

[1462] A "generative AI model" is an artificial intelligence-based model used to generate conversations with users, utilizing natural language processing techniques.

[1463] "Voice input" refers to the voice that the user utters into the terminal, and is used to convert this into text data and send it to the server.

[1464] "Text-to-text" refers to the process of converting voice input into text data, which is done using speech recognition technology.

[1465] "Emotion engine" refers to software or functionality for analyzing and acquiring emotional information from the user's speech and voice.

[1466] "Response generation" refers to the process of using a generative AI model to create an appropriate response to the user based on the acquired text data and emotional information.

[1467] "Speech synthesis technology" refers to technology for converting generated response text into voice data and providing it to the user aloud.

[1468] "Conversation history for each user" is a record of past conversations with a specific user, and is used to personalize the generative AI model.

[1469] "Feedback" refers to opinions and thoughts provided by users regarding a conversation, and is used to improve the system.

[1470] "Database" refers to an information storage system for saving and managing conversation history, feedback, emotional data, etc. with users.

[1471] MODE FOR CARRYING OUT THE INVENTION

[1472] The embodiment of this invention relates to a conversation service for dementia prevention that utilizes a generative model and an emotion engine, and aims to provide an environment where elderly people can enjoy conversation on a daily basis, and promote dementia prevention through brain activation and emotion recognition. Specific program processing and details of the embodiment are described below.

[1473] Server Processing

[1474] The server first loads a generative AI model (e.g., GPT-3). Next, it reads each user's conversation history from a database and initializes the generative AI model based on that data. This initialization enables personalized responses that reflect the user's past conversation data.

[1475] When a user sends voice input from a device, the server converts the voice data into text and receives it. At the same time, it uses an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the user's emotions and obtain emotional information. Based on the analysis results and emotional information, the server uses a generative AI model to generate an appropriate response and sends the response in text format to the device.

[1476] After the conversation is over, the server stores the user's feedback and emotional data in a database, which can later be used to improve the generative AI model and emotion engine.

[1477] Terminal handling

[1478] The device receives the user's voice input and converts it into text data using voice recognition technology (e.g., Google Cloud Speech-to-Text). The converted text data is then sent from the device to a server. This text data also includes identification information such as the user ID.

[1479] When the device receives the response text from the server, it uses speech synthesis technology (e.g., Amazon Polly) to convert the received text into speech and provides it to the user.

[1480] User Roles

[1481] The user initiates a conversation by speaking to the device. At this time, they can verbally express specific questions or requests. They listen to the voice responses provided by the device and provide feedback as needed. This feedback is used to improve the quality of future conversations.

[1482] Specific conversation examples

[1483] User: "Assistant, what's the weather forecast for this week?"

[1484] Terminal: "Send a request to the server."

[1485] Server: Received text from user ID 456 asking "What's the weather forecast this week?" and used the emotion engine to recognize "expectation" from the user's voice. Using the generative AI model, generated the response: "There will be many sunny days this week, with a maximum temperature of 25 degrees. Looking forward to it!"

[1486] Device: "There will be lots of sunny days this week, with a high of 25 degrees. Looking forward to it!"

[1487] User: "Thanks, that's very helpful."

[1488] This concludes the description of an embodiment of a conversation system using a generative model and an emotion engine. This system allows elderly people to enjoy conversations with others on a daily basis, and contributes to the prevention of dementia through brain activation and emotion recognition.

[1489] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1490] Program processing flow

[1491] Server Processing

[1492] Step 1: Initializing the generative AI model

[1493] How it works: The server first loads a generative AI model (e.g., GPT-3) that is used to reflect the user's unique past conversation history.

[1494] Input: Conversation history data for each user retrieved from the database.

[1495] Data processing and calculation: The server analyzes conversation history data and applies it to a generative AI model to customize it for each user.

[1496] Output: An initialized generative AI model.

[1497] Step 2: Receiving voice input and converting it to text

[1498] What happens: The device receives the user's voice input and converts it into text using speech recognition technology (e.g., Google Cloud Speech-to-Text).

[1499] Input: User's voice input.

[1500] Data processing and data calculation: Speech to text conversion using speech recognition technology.

[1501] Output: Text data (e.g. "Tell me the weather").

[1502] Step 3: Send a text

[1503] Specific operation: The terminal sends the converted text data along with the user ID to the server.

[1504] Input: Text data and user ID.

[1505] Data processing and data calculation: Formation of data structures for sending text data and identification information to the server.

[1506] Output: The text data sent to the server and the user ID.

[1507] Step 4: Text and sentiment analysis

[1508] Specific operation: The server analyzes the received text data and obtains the user's emotional information using an emotion engine (e.g., IBM Watson Tone Analyzer).

[1509] Input: Received text data.

[1510] Data processing and data calculation: Sentiment analysis and text content analysis using emotion engines.

[1511] Output: Analysis results and user sentiment information.

[1512] Step 5: Generate a response

[1513] How it works: The server uses a generative AI model to generate an appropriate response based on the analysis results and emotional information.

[1514] Input: Analysis results and emotion information.

[1515] Data processing and data calculation: Response generation using generative AI models.

[1516] Output: Response text (e.g. "Today the weather is sunny and the temperature is 20 degrees. It looks like it's going to be a great day!").

[1517] Step 6: Sending a Response

[1518] Specific operation: The server sends the generated response text to the terminal.

[1519] Input: The response text.

[1520] Data processing and data calculation: Transfer of response text.

[1521] Output: Sending the response text from the server to the terminal.

[1522] Step 7: Transcribing and outputting responses

[1523] Specific operation: The device converts the received response text into speech using speech synthesis technology (e.g., Amazon Polly) and provides it to the user.

[1524] Input: The response text.

[1525] Data processing and data arithmetic: Response text to speech conversion.

[1526] Output: A spoken response (e.g. "Today the weather is sunny and the temperature is 20 degrees. It's going to be a great day!").

[1527] Step 8: Collect and store feedback

[1528] Specific operation: After the conversation ends, the server stores the user's feedback and emotion data in a database.

[1529] Input: User feedback and sentiment data.

[1530] Data processing and data computation: Analysis and storage of feedback and emotion data.

[1531] Output: Feedback and sentiment data stored in a database.

[1532] User Action

[1533] Step 1: Start a conversation

[1534] Specific actions: The user speaks to the device and verbally communicates a specific question or request.

[1535] Input: User's voice.

[1536] Data processing and data arithmetic: Shaping what we say.

[1537] Output: Voice input to the device (e.g., "Assistant, what's the weather today?").

[1538] Step 2: Acceptance of responses and feedback

[1539] Specific operation: The user listens to the voice response from the device and provides appropriate feedback based on the content.

[1540] Input: A voice response from the device.

[1541] Data processing and data arithmetic: Shaping feedback based on response content.

[1542] Output: User feedback (e.g. "Thanks, that's very helpful").

[1543] These are the specific processing steps of this system's program. This allows elderly people to enjoy conversations with others on a daily basis, and contributes to the prevention of dementia through brain activation and emotional recognition.

[1544] (Application example 2)

[1545] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1546] To enable elderly people to use food delivery services smoothly, simple voice-activated operation is required. However, conventional systems have difficulty responding to individual emotions, resulting in a poor user experience. Furthermore, they are unable to generate personalized suggestions or responses based on order history, which may lead to a decrease in user satisfaction. There is a need to provide a system that can solve these issues and enable elderly people to use food delivery services comfortably.

[1547] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1548] In this invention, the server includes means for disabling the generative model and initializing a user-specific model for conversation generation based on organized data including each user's conversation history, means for converting speech input from the user into text and sending the text to the server, means for generating appropriate responses using the generative model based on the text received by the server, means for using speech synthesis technology to output the generated responses to the user, means for collecting feedback from the user and storing the feedback in a database for use in improving future conversation generation, means for using an emotion engine to ensure that suggested responses reflect the user's emotional state, and means for suggesting appropriate menu items to the user based on emotions and past ordering history. This allows for the generation of personalized responses based on the user's emotions, enabling elderly people to smoothly use food delivery services in an emotionally satisfying manner.

[1549] 1. A "generative model" is a machine learning algorithm that generates appropriate responses based on user input data.

[1550] 2. "Conversation history" is a record of past conversations a user has had, and is data used to provide personalized support in future conversations.

[1551] 3. "Voice input" refers to voice data uttered by a user through a device such as a microphone.

[1552] 4. "Convert to text" is the process of converting voice input into text data.

[1553] 5. “Server” means a computer system that hosts Generative Models and processes data received from Users.

[1554] 6. An "emotion engine" is an algorithm that analyzes a user's emotions from voice and text data and identifies their emotional state.

[1555] 7. "Speech synthesis technology" means technology that converts text data into voice data and is used to communicate generated responses to the user by voice.

[1556] 8. "Feedback" means the responses or opinions provided by a user and any emotional data associated with those responses.

[1557] 9. "Order History" means a record of past orders placed by a User and is used to make recommendations for future orders.

[1558] 10. "Menu Suggestions" are a list of foods or items that the system suggests to the user based on past ordering history and the user's emotional state.

[1559] This invention relates to a food delivery ordering support system for seniors that utilizes a generative model and an emotion engine. The system enables seniors to easily order food delivery using voice and provides personalized suggestions through emotion analysis to enhance the user experience.

[1560] System Overview

[1561] Hardware

[1562] Smartphones: Devices used by seniors that support voice input and output.

[1563] Server: A computer system that hosts the generative model and emotion engine and processes data received from users.

[1564] software

[1565] Generative AI model (GPT-2): A machine learning algorithm used to generate appropriate responses to user input.

[1566] Emotion engine: Software for analyzing user emotions from voice and text data to identify emotional states. For example, we use the EmotionAnalyzer library.

[1567] Speech recognition technology: Technology to convert the elderly's voice input into text. As an example, we will use the Google Speech-to-Text API.

[1568] Speech synthesis technology: Technology to convert responses from the server into voice data and convey them to the user. As an example, we will use the Google Text-to-Speech API.

[1569] System processing flow

[1570] Server Processing

[1571] 1. Model initialization: The server loads the generative model, reads each user's order history, and initializes the model based on that data. This allows for personalized responses that reflect the user's past order data.

[1572] 2. Receiving and analyzing voice input: The terminal receives voice input from the elderly person and analyzes the converted text. At the same time, it uses an emotion engine to analyze the user's emotions and obtain emotional information.

[1573] 3. Response and suggestion generation: Based on the analysis results and sentiment information, a generative model is used to generate appropriate responses and suggestions. For example, suggesting appropriate menu items based on the user's sentiment and past ordering history.

[1574] 4. Feedback collection: After the conversation is over, the server receives feedback and sentiment data from the user and stores it in a database, which can improve the quality of responses in the future.

[1575] Terminal handling

[1576] 1. Voice input and text conversion: The device receives the voice spoken by the elderly person and converts it into text data using voice recognition technology.

[1577] 2. Sending the text data: The converted text data is sent to the server, including identification information such as the user ID.

[1578] 3. Receiving response and outputting voice: Receives the response text from the server, converts it into voice using speech synthesis technology, and plays it back to the user.

[1579] User Roles

[1580] 1. Start the conversation: The elderly person starts speaking to the device and placing an order by voice.

[1581] 2. Response acceptance and feedback: The elderly person listens to the response from the device, confirms the order, and provides feedback. The feedback is used to improve the quality of response generation in future.

[1582] Specific examples

[1583] User: "Can you help me place an order?"

[1584] Terminal: "Of course. What would you like to order today?"

[1585] User: "I want pizza."

[1586] Server: Based on user ID 123's order history, suggest a menu item that matches their order history, saying, "How about pizza? Margherita or pepperoni are also delicious. Which would you like?"

[1587] Terminal: "How about some pizza? Margherita or pepperoni are both delicious. Which would you like?"

[1588] User: "Margherita, please."

[1589] Terminal: "Okay, I'll order a Margherita pizza!"

[1590] Prompt Sentence Examples

[1591] User input: "I want pizza."

[1592] Sentiment analysis result: “Expectation”

[1593] Suggestion generation prompt: "The user has a sense of anticipation. Suggest Margherita pizza or pepperoni pizza based on their past orders."

[1594] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1595] Step 1:

[1596] Initializing the Server

[1597] The server loads the generative model and initializes it with each user's order history, allowing it to generate user-specific responses based on past data.

[1598] Input: User ID and order history data

[1599] Processing: The server analyzes the data for each user and adjusts the generative model

[1600] Output: An initialized generative model

[1601] Step 2:

[1602] Receiving and converting voice input

[1603] The user speaks to the smartphone to input voice, which the device receives and converts into text using voice recognition technology.

[1604] Input: User voice input

[1605] Processing: Using speech recognition technology (e.g., Google Speech-to-Text API) to convert voice data into text data

[1606] Output: Text data

[1607] Step 3:

[1608] Sending text data

[1609] The device sends the converted text data to the server, which also includes identification information such as the user ID.

[1610] Input: Text data and user ID

[1611] Process: Send the text data and user ID to the server

[1612] Output: The input text received by the server

[1613] Step 4:

[1614] Emotion analysis

[1615] The server analyzes the user's emotions using an emotion engine based on the received text data.

[1616] Input: Text data

[1617] Processing: Analyze user emotions using an emotion engine (e.g., EmotionAnalyzer library)

[1618] Output: Emotional state data

[1619] Step 5:

[1620] Generate responses and suggestions

[1621] Generative AI models are used to generate appropriate responses and suggestions based on sentiment information and past order history.

[1622] Input: Text data, emotional state data, order history data

[1623] Processing: Generative AI models generate appropriate responses and menu suggestions based on the input data.

[1624] Output: Response text data

[1625] Step 6:

[1626] Audio output of the response

[1627] The terminal receives the response text data from the server, converts it into voice data using voice synthesis technology, and plays it back to the user.

[1628] Input: Response text data

[1629] Processing: Convert text data into audio data using speech synthesis technology (e.g., Google Text-to-Speech API).

[1630] Output: The audio data to be played back to the user

[1631] Step 7:

[1632] Gathering feedback

[1633] After the conversation is finished, the user provides feedback to the terminal, which sends this feedback to the server, which stores it in a database.

[1634] Input: User feedback

[1635] Processing: The server records the feedback and emotional state data and uses it to improve the next conversation generation.

[1636] Output: Data for an improved generative model

[1637] As a result of the above, the system allows seniors to easily order food delivery using voice commands and provides personalized service through sentiment analysis.

[1638] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1639] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1640] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1641] [Fourth embodiment]

[1642] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1643] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1644] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1645] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1646] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1647] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1648] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1649] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1650] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1651] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1652] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1653] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1654] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1655] The embodiment of the present invention relates to a conversation service for preventing dementia that utilizes a generative model, and the specific program processing and explanation in natural language are provided below.

[1656] An embodiment of a conversation system using a generative model

[1657] The purpose of this system is to provide an environment where elderly people can enjoy daily conversations and promote brain activation. The main components and operation details are shown below.

[1658] Server Processing

[1659] 1. Initializing the AI ​​model

[1660] The server first loads the generative AI model, reads each user's conversation history, and initializes the model based on that information. This enables personalized responses that reflect the user's past conversation data.

[1661] Example: The server initialized the generative AI model based on the conversation history of user ID 123.

[1662] 2. Receiving and analyzing user requests

[1663] When a user sends voice input from a device, the server receives it, converts it into text, and performs context analysis to understand the user's intent and requests.

[1664] Example: The server receives the text "How's the weather today?" and interprets it as a question about the weather.

[1665] 3. Generating responses using a generative model

[1666] Based on the analysis results, the server uses a generative AI model to generate an appropriate response that provides the best answer to the user's question.

[1667] Example: The server uses a generative AI model to generate the response, "Today's weather is sunny and the temperature is 25 degrees."

[1668] 4. Gather feedback and improve the model

[1669] After the conversation is over, the server receives feedback from the user and stores it in a database, which helps improve the quality of the generative AI model.

[1670] Example: The server stores the feedback "the response was helpful" in its database.

[1671] Terminal handling

[1672] 1. Voice to Text

[1673] The device receives the user's voice and converts it into text using voice recognition technology.

[1674] Example: A user says "What's the weather?" which is converted by the device to "What's the weather?"

[1675] 2. Sending a text

[1676] The terminal sends the converted text to the server.

[1677] Example: The device sends the text "Tell me the weather" to the server.

[1678] 3. Receiving a response from the server and outputting audio

[1679] The device receives the response text from the server and converts it internally into speech using speech synthesis technology, which is then played back to the user.

[1680] Example: The device spoke the response to the user: "Today's weather is sunny and the temperature is 25 degrees."

[1681] User Roles

[1682] 1. Start a conversation

[1683] The user starts a conversation by speaking to the device, asking specific questions or making requests.

[1684] Example: A user says, "Alexa, what's the weather like today?"

[1685] 2. Response acceptance and feedback

[1686] The user listens to the responses provided by the device and provides appropriate feedback, which is used to improve the quality of future conversations.

[1687] Example: The user provides feedback saying, "Thanks, now I can go for a walk."

[1688] Specific conversation examples

[1689] User: "Alexa, what's the weather forecast for this week?"

[1690] Terminal: "Send a request to the server."

[1691] Server: Received text from user ID 456 asking "What's the weather forecast this week?" and used the generative model to generate the response "There will be many sunny days this week, with a maximum temperature of 25 degrees."

[1692] Device: "There will be many sunny days this week, with a high of 25 degrees," the user is told.

[1693] User: "Thanks, that's very helpful."

[1694] This concludes the description of an embodiment of a conversation system using a generative model. This system allows elderly people to enjoy conversations with others on a daily basis, promoting brain activity and contributing to the prevention of dementia.

[1695] The processing flow will be explained below.

[1696] Step 1:

[1697] The server initializes the generative model. Specifically, it reads the conversation history of each user, customizes the generative model based on that data, and then initializes it.

[1698] Step 2:

[1699] The user verbally inputs questions and requests into the device in a natural conversational style.

[1700] Step 3:

[1701] The device receives the user's voice and converts it into text data using voice recognition technology. If the voice input is "Tell me the weather," the text will also be "Tell me the weather."

[1702] Step 4:

[1703] The terminal sends the converted text to the server, which also includes identification information such as the user ID.

[1704] Step 5:

[1705] The server receives the text data sent from the device, analyzes the received data, and performs context analysis to understand the user's intent.

[1706] Step 6:

[1707] The server uses the generative model to generate an appropriate response based on the analysis results, such as "Today's weather is sunny and the temperature is 20 degrees."

[1708] Step 7:

[1709] The server generates a response and sends it back to the terminal in text format, which contains the response content.

[1710] Step 8:

[1711] The terminal generates voice data based on the received response text using speech synthesis technology and outputs the voice to the user.

[1712] Step 9:

[1713] The user receives a voice response from the terminal and provides feedback as needed, such as "Thank you, that was helpful."

[1714] Step 10:

[1715] The terminal transmits the feedback from the user back to the server, and the transmitted data includes the content of the user's feedback.

[1716] Step 11:

[1717] The server analyzes the received feedback data and stores it in a database for future improvements to the generative model. This feedback data is used as reference information for future conversation generation.

[1718] Example 1

[1719] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1720] To contribute to the prevention of dementia among the elderly, it is important to provide an environment where people can enjoy everyday conversations. However, providing conversations that meet individual needs requires advanced natural language processing technology and responses that reflect the user's past conversation history. Furthermore, to improve the quality of the generated responses, a system is needed to efficiently collect user feedback and use it to improve the model. However, current technology does not provide a system that can consistently achieve these goals.

[1721] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1722] In this invention, the server includes means for loading a generative AI model and reading each user's conversation history to initialize the model, means for converting voice input from the user into text and sending the text to the server, means for the server to perform context analysis of the received text and generate an appropriate response using the generative AI model, means for using speech synthesis technology to output the generated response to the user, and means for collecting feedback from users and storing the feedback in a database for use in improving future conversation generation, thereby providing conversations tailored to individual needs and enabling continuous improvement of the model using the feedback.

[1723] A "generative AI model" is a machine learning model trained to generate text and data using artificial intelligence (AI).

[1724] "Conversation history" is a record of past conversations that a particular user has had, data that can be used to personalize future responses.

[1725] "Speech recognition" is the technology that analyzes voice data and converts it into corresponding text.

[1726] "Context analysis" is a natural language processing technique used to understand the content and intent of input text.

[1727] "Speech synthesis technology" is a technology that converts text data into speech and generates artificially synthesized speech.

[1728] "Feedback" refers to ratings and comments on responses provided by users, and is information used to improve the system.

[1729] A "database" is a collection of information that stores data in an organized manner and can be efficiently accessed when needed.

[1730] "Natural language processing technology" is technology for understanding, interpreting, and generating human language.

[1731] "Machine learning" is a technique that uses data to improve algorithms for pattern recognition and prediction.

[1732] This invention relates to a conversation system for dementia prevention using a generative AI model. In particular, it provides a mechanism for initializing the generative model based on an individual conversation history, converting speech input into text, generating responses using the generative model, synthesizing speech output, and collecting and learning feedback.

[1733] Overall system configuration

[1734] This system mainly consists of three main components: a server, a terminal, and a user. The server has the generative AI model, the terminal is responsible for speech recognition and speech synthesis, and the user has a natural conversation with the system. Each component is described in detail below.

[1735] Server Roles

[1736] Loading and initializing the generative AI model

[1737] The server loads a generative AI model (e.g., GPT-4) and runs the model using a high-performance GPU. Each user's conversation history is retrieved from a database and input into the model for initialization. This enables personalized responses that reflect the user's past conversation data.

[1738] Receiving and analyzing user requests

[1739] The server receives voice input from the device and converts it to text using a speech recognition engine (e.g., Google Speech-to-Text API). The converted text is then contextually analyzed using a generative AI model to understand the user's intent.

[1740] Response generation using generative models

[1741] Based on the analysis results, the server uses a generative AI model to generate an appropriate response to the user's question or request. In this process, the user's question is given as an input prompt, and the model generates an appropriate answer.

[1742] Example: In response to a user question, "What's the weather forecast this week?", the generative AI model generates the response, "There will be many sunny days this week, with a maximum temperature of 25 degrees."

[1743] Gathering feedback and improving the model

[1744] After the conversation, the server receives feedback from the user and stores it in a database. The collected feedback is used to improve the quality of the generative AI model and for continuous learning.

[1745] Device Role

[1746] Voice to text conversion

[1747] The device collects the user's voice using a built-in microphone and converts it into text using voice recognition technology (e.g., Google Speech-to-Text API). This conversion process ensures that the system accurately conveys natural speech from the user.

[1748] Sending text and voice output

[1749] The device sends the converted text to the server and receives a response from the server. The received response text is converted into speech using speech synthesis technology (e.g., Amazon Polly) and conveyed to the user.

[1750] Example: The device will speak the response to the user, "There will be many sunny days this week, with a maximum temperature of 25 degrees."

[1751] User Roles

[1752] Initiating a conversation and accepting a response

[1753] The user initiates a conversation by speaking into the device, asking specific questions or making requests, listening to the responses provided by the device and providing feedback as needed.

[1754] For example, when a user says, "Assistant, what's the weather like today?", the system responds by providing an appropriate response.

[1755] Specific examples

[1756] 1. User: "Assistant, what's the weather forecast for this week?"

[1757] 2. Device: Converts speech to text and sends it to the server.

[1758] 3. Server: Input the text "What's the weather forecast this week?" into the generative AI model and generate a response.

[1759] 4. Server: Sends the response "There will be many sunny days this week, and the maximum temperature will be 25 degrees" to the terminal.

[1760] 5. Terminal: Converts the received text into speech and conveys it to the user.

[1761] 6. User: "Thanks, that would be great."

[1762] In this way, elderly people can enjoy conversation with someone on a daily basis, which can promote brain activation and contribute to the prevention of dementia.

[1763] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1764] Step 1: Load and initialize the AI ​​model

[1765] ------

[1766] The server loads the generative AI model and reads the conversation history for each user.

[1767] Input: The server retrieves a file from the database containing the user ID, the corresponding conversation history, and the generative AI model (e.g., GPT-4).

[1768] process:

[1769] The server loads the generative AI model using a high-performance GPU.

[1770] Next, we load the past conversation history from the database based on the user ID and use this history to initialize the model.

[1771] Output: The initialized generative AI model runs in memory and reflects the conversational context of each user.

[1772] Specific behavior:

[1773] The server loads the GPT-4 model into memory on a GPU server in the data center.

[1774] The server retrieves the past conversation log of user ID 123 from the database and incorporates it into the model.

[1775] Step 2: Convert voice to text

[1776] ------

[1777] The terminal converts the user's voice into text and sends it to the server.

[1778] Input: Voice data spoken by the user into the device.

[1779] process:

[1780] The device's built-in microphone collects the user's voice and converts it into text using a local or cloud-based speech recognition engine (e.g., Google Speech-to-Text API).

[1781] Output: Text data is generated and sent from the terminal to the server.

[1782] Specific behavior:

[1783] When a user says, "Alexa, what's the weather today?", the device records this speech and converts it into text: "Alexa, what's the weather today?"

[1784] Step 3: Contextual analysis of the text

[1785] ------

[1786] The server parses the received text and uses a generative AI model to generate an appropriate response.

[1787] Input: User's text data received by the server from the device.

[1788] process:

[1789] The server provides the generative AI model with the user's text as an input prompt, which the model analyzes internally and generates an appropriate response.

[1790] Natural language processing (NLP) technology is used for context analysis.

[1791] Output: You get the generated response text.

[1792] Specific behavior:

[1793] The server receives the text "What's the weather forecast this week?" and analyzes it to generate a response saying "There will be many sunny days this week, with a maximum temperature of 25 degrees."

[1794] Step 4: Text-to-speech response

[1795] ------

[1796] The terminal receives the response text from the server, converts it into voice, and conveys it to the user.

[1797] Input: Response text data sent from the server.

[1798] process:

[1799] The device uses speech synthesis technology (e.g., Amazon Polly) to convert the received text data into speech.

[1800] Output: Audio data is generated and played through the speaker.

[1801] Specific behavior:

[1802] The device converts the response "There will be many sunny days this week, with a maximum temperature of 25 degrees" into speech and conveys it to the user.

[1803] Step 5: Gather feedback

[1804] ------

[1805] The server stores user feedback in a database to help improve the quality of the model.

[1806] Input: Feedback data provided by the user.

[1807] process:

[1808] The server receives the feedback and stores it in a database.

[1809] The collected feedback is used to retrain or tune the generative AI model.

[1810] Output: The saved feedback data.

[1811] Specific behavior:

[1812] The user provides feedback such as "Thanks, now I can go for a walk," and the server stores this in a database, which is then used to improve the model.

[1813] This is the specific program processing flow of this system. This allows elderly people to enjoy conversations with others on a daily basis, promoting brain activity and helping to prevent dementia.

[1814] (Application example 1)

[1815] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1816] In modern society, cognitive decline among the elderly is a serious problem. In particular, lack of communication inhibits brain activation and contributes to the accelerated progression of dementia. However, many elderly people have limited opportunities to enjoy conversation with others on a daily basis, and effective methods for providing such opportunities are needed. Even community facilities and cafes for the elderly lack systems that provide consistent, high-quality conversation partners. Technology is needed to solve this problem and improve the quality of life for the elderly.

[1817] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1818] In this invention, the server includes means for disabling the generative model and initializing a model for user-specific conversation generation based on organized data including each user's conversation history, means for converting speech input from the user into text and sending the text to the server, means for generating an appropriate response using the generative model based on the text received by the server, means for using speech synthesis technology to output the generated response to the user, means for collecting feedback from the user and storing the feedback in a database for use in improving future conversation generation, and means for providing natural conversation with users at a community facility for the elderly using a smart device linked to the generative model. This allows elderly people to enjoy daily conversations even in community facilities and cafes, thereby activating their brains.

[1819] A "generative model" is an algorithm that uses artificial intelligence to generate text, images, etc.

[1820] "Conversation history for each user" refers to a record of conversations that each user has had in the past.

[1821] A "means for initializing a model" is a method for starting the operation of a generative model based on specific conditions or data.

[1822] "Means for converting voice input to text" refers to technology that converts the voice spoken by a user into text information.

[1823] "Means for transmitting to the server" refers to a method for transmitting data from the terminal to the server.

[1824] The "means for generating an appropriate response" is a technology for generating a response for forming a natural conversation based on a user's input.

[1825] "Means using speech synthesis technology" refers to technology for converting text data into speech and outputting it.

[1826] "Feedback collection methods" are methods for collecting user-provided ratings and opinions.

[1827] "Means for storing data in a database" refers to the technology for recording collected data so that it can be used later.

[1828] "Community facilities" are social institutions used by the elderly to enjoy social interactions and activities.

[1829] "Smart terminal" is a general term for highly functional portable devices, including smartphones and tablets.

[1830] "Natural conversation" refers to interaction with artificial intelligence that is similar to a human conversation.

[1831] "Brain activation" refers to stimulation and activities that enhance brain function.

[1832] This invention is a conversation support system for elderly community facilities that utilizes generative models. The aim is to provide an environment where elderly people can enjoy daily conversations and promote brain activation. The main components and operation are detailed below.

[1833] Server Processing

[1834] Initializing the AI ​​model

[1835] The server first loads the generative AI model, reads each user's conversation history, and initializes the model based on that. This enables personalized responses that reflect the user's past conversation data. An example of a generative AI model used is OpenAI's GPT-3.

[1836] Receiving and analyzing user requests

[1837] When a user sends voice input from their device, the server receives it and performs context analysis on the converted text. This allows the server to understand the user's intent and requests. The specific software used for the voice recognition technology is the Google Speech-to-Text API.

[1838] Generating responses using generative models

[1839] Based on the analysis results, the server uses a generative AI model to generate an appropriate response, which provides the best answer to the user's question.

[1840] Gathering feedback and improving the model

[1841] After the conversation is over, the server receives feedback from the user and stores it in a database, which helps improve the quality of the generative AI model.

[1842] Terminal handling

[1843] Voice to text conversion

[1844] The device receives the user's voice and converts it into text. This process uses voice recognition technology. Specific hardware used is a smart speaker (e.g., Amazon Echo) or a tablet (e.g., iPad).

[1845] Sending a text

[1846] The terminal sends the converted text to the server using a communication protocol such as HTTPS.

[1847] Receiving a response from the server and outputting audio

[1848] The device receives the response text from the server, converts it internally into speech using speech synthesis technology, and uses the Google Text-to-Speech API to play the converted speech for the user.

[1849] User Roles

[1850] Start a conversation

[1851] The user starts a conversation by speaking to the device, asking specific questions or making requests. For example, "What's the weather like today?" or "Tell me the latest news."

[1852] Response acceptance and feedback

[1853] The user listens to the responses provided by the device and provides appropriate feedback, which is used to improve the quality of future conversations.

[1854] Specific conversation examples

[1855] User: "What kind of music do you like today?"

[1856] Terminal: "Send a request to the server."

[1857] Server: Received text from user ID 123 asking, "What kind of music do you like today?" and used the generative model to generate the response, "I've been enjoying classical music lately. I think Beethoven's symphonies are particularly wonderful."

[1858] Device: "I've been enjoying classical music lately. I think Beethoven's symphonies are particularly wonderful."

[1859] User: "Thanks, that's very helpful."

[1860] This concludes the description of an embodiment of a conversation support system for elderly community facilities that uses a generative model. This system allows elderly people to enjoy conversations with others on a daily basis, promoting brain activation and contributing to the prevention of dementia.

[1861] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1862] Step 1:

[1863] Receiving voice input

[1864] Subject: Device

[1865] How it works: The device receives audio spoken by the user.

[1866] Input: User speech (e.g., "What kind of music do you like today?")

[1867] Data processing and calculation: The microphone installed on the device converts the voice data into a digital signal.

[1868] Output: Digital audio data

[1869] Step 2:

[1870] Converting audio data to text

[1871] Subject: Device

[1872] How it works: The device converts digital voice data into text.

[1873] Input: Digital audio data

[1874] Data processing and calculation: The device's voice recognition software (e.g., Google Speech-to-Text API) converts the voice data into text information.

[1875] Output: Text data (e.g., "What kind of music do you like today?")

[1876] Step 3:

[1877] Sending text data to the server

[1878] Subject: Device

[1879] Operation: The terminal sends the converted text data to the server.

[1880] Input: Text data

[1881] Data processing and calculation: Text data is sent to the server using the HTTPS protocol.

[1882] Output: Request data to the server

[1883] Step 4:

[1884] Contextual analysis of text data

[1885] Subject: Server

[1886] How it works: The server analyzes the received text data and extracts the user's intent and requests.

[1887] Input: Text data

[1888] Data processing and calculation: The server's context analysis engine analyzes the text data and identifies the intent and request.

[1889] Output: Context analysis results (e.g., identifying that the question is about music)

[1890] Step 5:

[1891] Generating an appropriate response

[1892] Subject: Server

[1893] How it works: The server generates a response using a generative AI model (e.g., OpenAI GPT-3) based on the contextual analysis results.

[1894] Input: Context analysis results

[1895] Data processing and calculation: The generative AI model generates prompt sentences and generates optimal responses.

[1896] Output: Response text data (e.g., "I've been enjoying classical music lately. I think Beethoven's symphonies are particularly wonderful.")

[1897] Step 6:

[1898] Terminal reception of response data

[1899] Subject: Device

[1900] Operation: The terminal receives the response text data sent from the server.

[1901] Input: Response text data

[1902] Data processing and calculation: The terminal receives the response data using the HTTPS protocol.

[1903] Output: Response text data (e.g., "I've been enjoying classical music lately. I think Beethoven's symphonies are particularly wonderful.")

[1904] Step 7:

[1905] Response text-to-speech

[1906] Subject: Device

[1907] Operation: The device converts the received response text data into speech.

[1908] Input: Response text data

[1909] Data processing and calculation: The device's speech synthesis software (e.g., Google Text-to-Speech API) converts the text data into voice data.

[1910] Output: Response audio data

[1911] Step 8:

[1912] Response voice output to the user

[1913] Subject: Device

[1914] Action: The device plays the audio data and communicates it to the user.

[1915] Input: Response audio data

[1916] Data processing and calculation: Plays audio through the device's built-in speaker.

[1917] Output: Response speech (e.g., "I've been enjoying classical music lately. I think Beethoven's symphonies are particularly wonderful.")

[1918] Step 9:

[1919] Gathering feedback

[1920] Subject: Device

[1921] Action: The device receives feedback from the user.

[1922] Input: User feedback

[1923] Data processing and calculation: The device converts voice input back into text or collects feedback via touch input, etc.

[1924] Output: Feedback text data

[1925] Step 10:

[1926] Feedback data sent to server and stored

[1927] Subject: Terminal and Server

[1928] Operation: The device sends the feedback text data to the server, which stores it in a database.

[1929] Input: Feedback text data

[1930] Data processing and calculation: The terminal sends the feedback text data to the server, and the server stores the received feedback data in a database.

[1931] Output: Stored feedback data

[1932] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1933] The embodiment of the present invention relates to a conversation service for preventing dementia that utilizes a generative model and an emotion engine, and the specific program processing and explanation in natural language are provided below.

[1934] An embodiment of a conversation system using a generative model and an emotion engine

[1935] The system aims to provide an environment where elderly people can enjoy daily conversations and promote the prevention of dementia through brain activation and emotion recognition. The main components and operation details are shown below.

[1936] Server Processing

[1937] 1. Initializing the AI ​​model

[1938] The server loads the generative AI model, reads each user's conversation history, and initializes the model based on that data, enabling personalized responses that reflect the user's past conversation data.

[1939] Example: The server initialized the generative AI model based on the conversation history of user ID 123.

[1940] 2. Receiving user requests and analyzing their sentiment

[1941] When a user sends a voice input from a terminal, the server receives it, converts it into text, analyzes the content, and simultaneously analyzes the user's emotions using an emotion engine to obtain emotion information.

[1942] Example: The server receives the text "How's the weather today?" and then uses an emotion engine to recognize the emotion "happy" from the user's voice.

[1943] 3. Generating responses using a generative model

[1944] Based on the analysis results and emotional information, the server uses a generative AI model to generate an appropriate response that reflects the user's current emotions.

[1945] Example: The server used a generative AI model to generate the response, "Today's weather is sunny and the temperature is 20 degrees. It looks like it's going to be a great day!"

[1946] 4. Gather feedback and improve the model

[1947] After the conversation is over, the server receives user feedback and emotion data and stores it in a database. The collected data will be used to improve the quality of the generative AI model and emotion engine.

[1948] Example: The server stores the feedback "the response was helpful" and the emotion data "fun" in a database.

[1949] Terminal handling

[1950] 1. Voice to Text

[1951] The device receives the user's voice and converts it into text data using voice recognition technology. If the voice input is "Tell me the weather," the text will also be "Tell me the weather."

[1952] Example: A user says "What's the weather?" which is converted by the device to "What's the weather?"

[1953] 2. Sending a text

[1954] The terminal sends the converted text to the server, which also includes identification information such as the user ID.

[1955] Example: The device sends the text "Tell me the weather" to the server.

[1956] 3. Receiving a response from the server and outputting audio

[1957] The device receives the response text from the server and converts it internally into speech using speech synthesis technology, which is then played back to the user.

[1958] Example: The device spoke the response to the user: "Today's weather is sunny and the temperature is 20 degrees. It looks like it's going to be a great day!"

[1959] User Roles

[1960] 1. Start a conversation

[1961] The user starts a conversation by speaking to the terminal, asking specific questions or making requests verbally.

[1962] Example: A user says, "Alexa, what's the weather like today?"

[1963] 2. Response acceptance and feedback

[1964] The user listens to the responses provided by the device and provides appropriate feedback, which is used to improve the quality of future conversations.

[1965] Example: The user provides feedback saying, "Thanks, now I can go for a walk."

[1966] Specific conversation examples

[1967] User: "Alexa, what's the weather forecast for this week?"

[1968] Terminal: "Send a request to the server."

[1969] Server: Received text from user ID 456 asking "What's the weather forecast this week?" and used the emotion engine to recognize "expectation" from the user's voice. Using the generative model, generated the response: "There will be many sunny days this week, with a maximum temperature of 25 degrees. Looking forward to it!"

[1970] Device: "There will be lots of sunny days this week, with a high of 25 degrees. Looking forward to it!"

[1971] User: "Thanks, that's very helpful."

[1972] This concludes the description of an embodiment of a conversation system using a generative model and an emotion engine. This system allows elderly people to enjoy conversations with others on a daily basis, and contributes to the prevention of dementia through brain activation and emotion recognition.

[1973] The processing flow will be explained below.

[1974] Step 1:

[1975] The server initializes the generative model. Specifically, it reads the conversation history of each user, customizes the generative model based on that data, and then initializes it.

[1976] Step 2:

[1977] The user verbally inputs questions and requests into the device in a natural conversational style.

[1978] Step 3:

[1979] The device receives the user's voice and converts it into text data using voice recognition technology. For example, if the voice input is "Tell me the weather," the text will also be "Tell me the weather."

[1980] Step 4:

[1981] The terminal sends the converted text to the server, which also includes identification information such as the user ID.

[1982] Step 5:

[1983] The server receives the text data sent from the device, analyzes the received data, and performs context analysis to understand the user's intent.

[1984] Step 6:

[1985] The server uses an emotion engine to extract the user's emotion from the speech along with context analysis. The emotion engine analyzes the tone and rate of the speech to identify the user's emotion.

[1986] Step 7:

[1987] The server uses a generative model to generate an appropriate response based on the analysis and emotional information, for example, taking into account a question about the weather and the emotion "fun" to tailor the response.

[1988] Step 8:

[1989] The server sends the generated response back to the terminal in text format. The response content is included in the sent data.

[1990] Step 9:

[1991] The device uses speech synthesis technology to generate voice data based on the received response text and outputs it to the user. For example, it might output, "Today's weather is sunny, and the temperature is 20 degrees. It looks like it's going to be a nice day today!"

[1992] Step 10:

[1993] The user receives a voice response from the terminal and provides feedback as needed, such as "Thank you, that was helpful."

[1994] Step 11:

[1995] The terminal transmits the feedback from the user to the server, and the transmitted data includes the content of the user's feedback.

[1996] Step 12:

[1997] The server analyzes the received feedback and emotion data and stores it in a database for future use in improving the generative model and emotion engine.

[1998] Example 2

[1999] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2000] Brain activation and emotional recognition through everyday conversations are important for preventing dementia in the elderly. However, it is difficult for elderly people to find suitable conversation partners on a daily basis, and conventional conversation systems are insufficient in providing individualized responses that reflect the user's emotions and past conversation history, limiting their effectiveness in preventing dementia. The present invention solves these problems and provides technology that promotes dementia prevention while allowing elderly people to enjoy everyday conversations.

[2001] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for loading a generative AI model and initializing a user-specific model based on data including each user's conversation history; means for converting voice input from the user into text and sending the text to the server; means for analyzing the text received by the server and acquiring emotional information using an emotion engine; means for generating a response based on the analysis result and emotional information using the generative AI model; means for outputting the generated response to the user by voice using speech synthesis technology; and means for collecting feedback from the user and storing the feedback and emotional data in a database. This allows elderly people to enjoy individually tailored conversations on a daily basis, and promotes dementia prevention through brain activation and emotion recognition.

[2002] A "generative AI model" is an artificial intelligence-based model used to generate conversations with users, utilizing natural language processing techniques.

[2003] "Voice input" refers to the voice that the user utters into the terminal, and is used to convert this into text data and send it to the server.

[2004] "Text-to-text" refers to the process of converting voice input into text data, which is done using speech recognition technology.

[2005] "Emotion engine" refers to software or functionality for analyzing and acquiring emotional information from the user's speech and voice.

[2006] "Response generation" refers to the process of using a generative AI model to create an appropriate response to the user based on the acquired text data and emotional information.

[2007] "Speech synthesis technology" refers to technology for converting generated response text into voice data and providing it to the user aloud.

[2008] "Conversation history for each user" is a record of past conversations with a specific user, and is used to personalize the generative AI model.

[2009] "Feedback" refers to opinions and thoughts provided by users regarding a conversation, and is used to improve the system.

[2010] "Database" refers to an information storage system for saving and managing conversation history, feedback, emotional data, etc. with users.

[2011] MODE FOR CARRYING OUT THE INVENTION

[2012] The embodiment of this invention relates to a conversation service for dementia prevention that utilizes a generative model and an emotion engine, and aims to provide an environment where elderly people can enjoy conversation on a daily basis, and promote dementia prevention through brain activation and emotion recognition. Specific program processing and details of the embodiment are described below.

[2013] Server Processing

[2014] The server first loads a generative AI model (e.g., GPT-3). Next, it reads each user's conversation history from a database and initializes the generative AI model based on that data. This initialization enables personalized responses that reflect the user's past conversation data.

[2015] When a user sends voice input from a device, the server converts the voice data into text and receives it. At the same time, it uses an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the user's emotions and obtain emotional information. Based on the analysis results and emotional information, the server uses a generative AI model to generate an appropriate response and sends the response in text format to the device.

[2016] After the conversation is over, the server stores the user's feedback and emotional data in a database, which can later be used to improve the generative AI model and emotion engine.

[2017] Terminal handling

[2018] The device receives the user's voice input and converts it into text data using voice recognition technology (e.g., Google Cloud Speech-to-Text). The converted text data is then sent from the device to a server. This text data also includes identification information such as the user ID.

[2019] When the device receives the response text from the server, it uses speech synthesis technology (e.g., Amazon Polly) to convert the received text into speech and provides it to the user.

[2020] User Roles

[2021] The user initiates a conversation by speaking to the device. At this time, they can verbally express specific questions or requests. They listen to the voice responses provided by the device and provide feedback as needed. This feedback is used to improve the quality of future conversations.

[2022] Specific conversation examples

[2023] User: "Assistant, what's the weather forecast for this week?"

[2024] Terminal: "Send a request to the server."

[2025] Server: Received text from user ID 456 asking "What's the weather forecast this week?" and used the emotion engine to recognize "expectation" from the user's voice. Using the generative AI model, generated the response: "There will be many sunny days this week, with a maximum temperature of 25 degrees. Looking forward to it!"

[2026] Device: "There will be lots of sunny days this week, with a high of 25 degrees. Looking forward to it!"

[2027] User: "Thanks, that's very helpful."

[2028] This concludes the description of an embodiment of a conversation system using a generative model and an emotion engine. This system allows elderly people to enjoy conversations with others on a daily basis, and contributes to the prevention of dementia through brain activation and emotion recognition.

[2029] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2030] Program processing flow

[2031] Server Processing

[2032] Step 1: Initializing the generative AI model

[2033] How it works: The server first loads a generative AI model (e.g., GPT-3) that is used to reflect the user's unique past conversation history.

[2034] Input: Conversation history data for each user retrieved from the database.

[2035] Data processing and calculation: The server analyzes conversation history data and applies it to a generative AI model to customize it for each user.

[2036] Output: An initialized generative AI model.

[2037] Step 2: Receiving voice input and converting it to text

[2038] What happens: The device receives the user's voice input and converts it into text using speech recognition technology (e.g., Google Cloud Speech-to-Text).

[2039] Input: User's voice input.

[2040] Data processing and data calculation: Speech to text conversion using speech recognition technology.

[2041] Output: Text data (e.g. "Tell me the weather").

[2042] Step 3: Send a text

[2043] Specific operation: The terminal sends the converted text data along with the user ID to the server.

[2044] Input: Text data and user ID.

[2045] Data processing and data calculation: Formation of data structures for sending text data and identification information to the server.

[2046] Output: The text data sent to the server and the user ID.

[2047] Step 4: Text and sentiment analysis

[2048] Specific operation: The server analyzes the received text data and obtains the user's emotional information using an emotion engine (e.g., IBM Watson Tone Analyzer).

[2049] Input: Received text data.

[2050] Data processing and data calculation: Sentiment analysis and text content analysis using emotion engines.

[2051] Output: Analysis results and user sentiment information.

[2052] Step 5: Generate a response

[2053] How it works: The server uses a generative AI model to generate an appropriate response based on the analysis results and emotional information.

[2054] Input: Analysis results and emotion information.

[2055] Data processing and data calculation: Response generation using generative AI models.

[2056] Output: Response text (e.g. "Today the weather is sunny and the temperature is 20 degrees. It looks like it's going to be a great day!").

[2057] Step 6: Sending a Response

[2058] Specific operation: The server sends the generated response text to the terminal.

[2059] Input: The response text.

[2060] Data processing and data calculation: Transfer of response text.

[2061] Output: Sending the response text from the server to the terminal.

[2062] Step 7: Transcribing and outputting responses

[2063] Specific operation: The device converts the received response text into speech using speech synthesis technology (e.g., Amazon Polly) and provides it to the user.

[2064] Input: The response text.

[2065] Data processing and data arithmetic: Response text to speech conversion.

[2066] Output: A spoken response (e.g. "Today the weather is sunny and the temperature is 20 degrees. It's going to be a great day!").

[2067] Step 8: Collect and store feedback

[2068] Specific operation: After the conversation ends, the server stores the user's feedback and emotion data in a database.

[2069] Input: User feedback and sentiment data.

[2070] Data processing and data computation: Analysis and storage of feedback and emotion data.

[2071] Output: Feedback and sentiment data stored in a database.

[2072] User Action

[2073] Step 1: Start a conversation

[2074] Specific actions: The user speaks to the device and verbally communicates a specific question or request.

[2075] Input: User's voice.

[2076] Data processing and data arithmetic: Shaping what we say.

[2077] Output: Voice input to the device (e.g., "Assistant, what's the weather today?").

[2078] Step 2: Acceptance of responses and feedback

[2079] Specific operation: The user listens to the voice response from the device and provides appropriate feedback based on the content.

[2080] Input: A voice response from the device.

[2081] Data processing and data arithmetic: Shaping feedback based on response content.

[2082] Output: User feedback (e.g. "Thanks, that's very helpful").

[2083] These are the specific processing steps of this system's program. This allows elderly people to enjoy conversations with others on a daily basis, and contributes to the prevention of dementia through brain activation and emotional recognition.

[2084] (Application example 2)

[2085] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2086] To enable elderly people to use food delivery services smoothly, simple voice-activated operation is required. However, conventional systems have difficulty responding to individual emotions, resulting in a poor user experience. Furthermore, they are unable to generate personalized suggestions or responses based on order history, which may lead to a decrease in user satisfaction. There is a need to provide a system that can solve these issues and enable elderly people to use food delivery services comfortably.

[2087] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2088] In this invention, the server includes means for disabling the generative model and initializing a user-specific model for conversation generation based on organized data including each user's conversation history, means for converting speech input from the user into text and sending the text to the server, means for generating appropriate responses using the generative model based on the text received by the server, means for using speech synthesis technology to output the generated responses to the user, means for collecting feedback from the user and storing the feedback in a database for use in improving future conversation generation, means for using an emotion engine to ensure that suggested responses reflect the user's emotional state, and means for suggesting appropriate menu items to the user based on emotions and past ordering history. This allows for the generation of personalized responses based on the user's emotions, enabling elderly people to smoothly use food delivery services in an emotionally satisfying manner.

[2089] 1. A "generative model" is a machine learning algorithm that generates appropriate responses based on user input data.

[2090] 2. "Conversation history" is a record of past conversations a user has had, and is data used to provide personalized support in future conversations.

[2091] 3. "Voice input" refers to voice data uttered by a user through a device such as a microphone.

[2092] 4. "Convert to text" is the process of converting voice input into text data.

[2093] 5. “Server” means a computer system that hosts Generative Models and processes data received from Users.

[2094] 6. An "emotion engine" is an algorithm that analyzes a user's emotions from voice and text data and identifies their emotional state.

[2095] 7. "Speech synthesis technology" means technology that converts text data into voice data and is used to communicate generated responses to the user by voice.

[2096] 8. "Feedback" means the responses or opinions provided by a user and any emotional data associated with those responses.

[2097] 9. "Order History" means a record of past orders placed by a User and is used to make recommendations for future orders.

[2098] 10. "Menu Suggestions" are a list of foods or items that the system suggests to the user based on past ordering history and the user's emotional state.

[2099] This invention relates to a food delivery ordering support system for seniors that utilizes a generative model and an emotion engine. The system enables seniors to easily order food delivery using voice and provides personalized suggestions through emotion analysis to enhance the user experience.

[2100] System Overview

[2101] Hardware

[2102] Smartphones: Devices used by seniors that support voice input and output.

[2103] Server: A computer system that hosts the generative model and emotion engine and processes data received from users.

[2104] software

[2105] Generative AI model (GPT-2): A machine learning algorithm used to generate appropriate responses to user input.

[2106] Emotion engine: Software for analyzing user emotions from voice and text data to identify emotional states. For example, we use the EmotionAnalyzer library.

[2107] Speech recognition technology: Technology to convert the elderly's voice input into text. As an example, we will use the Google Speech-to-Text API.

[2108] Speech synthesis technology: Technology to convert responses from the server into voice data and convey them to the user. As an example, we will use the Google Text-to-Speech API.

[2109] System processing flow

[2110] Server Processing

[2111] 1. Model initialization: The server loads the generative model, reads each user's order history, and initializes the model based on that data. This allows for personalized responses that reflect the user's past order data.

[2112] 2. Receiving and analyzing voice input: The terminal receives voice input from the elderly person and analyzes the converted text. At the same time, it uses an emotion engine to analyze the user's emotions and obtain emotional information.

[2113] 3. Response and suggestion generation: Based on the analysis results and sentiment information, a generative model is used to generate appropriate responses and suggestions. For example, suggesting appropriate menu items based on the user's sentiment and past ordering history.

[2114] 4. Feedback collection: After the conversation is over, the server receives feedback and sentiment data from the user and stores it in a database, which can improve the quality of responses in the future.

[2115] Terminal handling

[2116] 1. Voice input and text conversion: The device receives the voice spoken by the elderly person and converts it into text data using voice recognition technology.

[2117] 2. Sending the text data: The converted text data is sent to the server, including identification information such as the user ID.

[2118] 3. Receiving response and outputting voice: Receives the response text from the server, converts it into voice using speech synthesis technology, and plays it back to the user.

[2119] User Roles

[2120] 1. Start the conversation: The elderly person starts speaking to the device and placing an order by voice.

[2121] 2. Response acceptance and feedback: The elderly person listens to the response from the device, confirms the order, and provides feedback. The feedback is used to improve the quality of response generation in future.

[2122] Specific examples

[2123] User: "Can you help me place an order?"

[2124] Terminal: "Of course. What would you like to order today?"

[2125] User: "I want pizza."

[2126] Server: Based on user ID 123's order history, suggest a menu item that matches their order history, saying, "How about pizza? Margherita or pepperoni are also delicious. Which would you like?"

[2127] Terminal: "How about some pizza? Margherita or pepperoni are both delicious. Which would you like?"

[2128] User: "Margherita, please."

[2129] Terminal: "Okay, I'll order a Margherita pizza!"

[2130] Prompt Sentence Examples

[2131] User input: "I want pizza."

[2132] Sentiment analysis result: “Expectation”

[2133] Suggestion generation prompt: "The user has a sense of anticipation. Suggest Margherita pizza or pepperoni pizza based on their past orders."

[2134] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2135] Step 1:

[2136] Initializing the Server

[2137] The server loads the generative model and initializes it with each user's order history, allowing it to generate user-specific responses based on past data.

[2138] Input: User ID and order history data

[2139] Processing: The server analyzes the data for each user and adjusts the generative model

[2140] Output: An initialized generative model

[2141] Step 2:

[2142] Receiving and converting voice input

[2143] The user speaks to the smartphone to input voice, which the device receives and converts into text using voice recognition technology.

[2144] Input: User voice input

[2145] Processing: Using speech recognition technology (e.g., Google Speech-to-Text API) to convert voice data into text data

[2146] Output: Text data

[2147] Step 3:

[2148] Sending text data

[2149] The device sends the converted text data to the server, which also includes identification information such as the user ID.

[2150] Input: Text data and user ID

[2151] Process: Send the text data and user ID to the server

[2152] Output: The input text received by the server

[2153] Step 4:

[2154] Emotion analysis

[2155] The server analyzes the user's emotions using an emotion engine based on the received text data.

[2156] Input: Text data

[2157] Processing: Analyze user emotions using an emotion engine (e.g., EmotionAnalyzer library)

[2158] Output: Emotional state data

[2159] Step 5:

[2160] Generate responses and suggestions

[2161] Generative AI models are used to generate appropriate responses and suggestions based on sentiment information and past order history.

[2162] Input: Text data, emotional state data, order history data

[2163] Processing: Generative AI models generate appropriate responses and menu suggestions based on the input data.

[2164] Output: Response text data

[2165] Step 6:

[2166] Audio output of the response

[2167] The terminal receives the response text data from the server, converts it into voice data using voice synthesis technology, and plays it back to the user.

[2168] Input: Response text data

[2169] Processing: Convert text data into audio data using speech synthesis technology (e.g., Google Text-to-Speech API).

[2170] Output: The audio data to be played back to the user

[2171] Step 7:

[2172] Gathering feedback

[2173] After the conversation is finished, the user provides feedback to the terminal, which sends this feedback to the server, which stores it in a database.

[2174] Input: User feedback

[2175] Processing: The server records the feedback and emotional state data and uses it to improve the next conversation generation.

[2176] Output: Data for an improved generative model

[2177] As a result of the above, the system allows seniors to easily order food delivery using voice commands and provides personalized service through sentiment analysis.

[2178] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2179] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2180] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2181] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2182] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2183] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2184] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2185] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2186] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2187] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2188] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2189] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2190] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2191] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2192] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2193] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2194] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2195] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2196] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2197] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2198] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2199] The following is further disclosed regarding the above embodiment.

[2200] (Claim 1)

[2201] A means for releasing the generative model and initializing a model for user-specific conversation generation based on organized data including the conversation history of each user;

[2202] means for converting speech input from a user into text and transmitting the text to a server;

[2203] a means for generating an appropriate response based on the received text by the server using a generative model;

[2204] means for using speech synthesis technology to audibly output the generated response to the user;

[2205] means for collecting user feedback and storing the feedback in a database for use in improving future conversation generation;

[2206] A system including:

[2207] (Claim 2)

[2208] 10. The system of claim 1, further comprising a context analysis means for analyzing user utterances and extracting conversational intent.

[2209] (Claim 3)

[2210] 10. The system of claim 1, further comprising: a learning means for continuously improving the generative model using feedback data from a user.

[2211] "Example 1"

[2212] (Claim 1)

[2213] A means to load the generative AI model and initialize the model by reading each user's conversation history.

[2214] means for converting speech input from a user into text and transmitting the text to a server;

[2215] a means for the server to contextually analyze the received text and generate an appropriate response using a generative AI model;

[2216] means for using speech synthesis technology to audibly output the generated response to the user;

[2217] means for collecting feedback from users and storing the feedback in a database for use in improving future conversation generation;

[2218] A system including:

[2219] (Claim 2)

[2220] 10. The system of claim 1, further comprising natural language processing techniques for analyzing user utterances and extracting conversational intent.

[2221] (Claim 3)

[2222] 10. The system of claim 1, further comprising machine learning means for continuously improving the generative AI model using feedback data from users.

[2223] "Application Example 1"

[2224] (Claim 1)

[2225] A means for releasing the generative model and initializing a model for user-specific conversation generation based on organized data including the conversation history of each user;

[2226] means for converting speech input from a user into text and transmitting the text to a server;

[2227] a means for generating an appropriate response based on the received text by the server using a generative model;

[2228] means for using speech synthesis technology to audibly output the generated response to the user;

[2229] means for collecting user feedback and storing the feedback in a database for use in improving future conversation generation;

[2230] A method for providing natural conversations with users in a community facility for the elderly using a smart device linked with a generative model;

[2231] A system including:

[2232] (Claim 2)

[2233] 10. The system of claim 1, further comprising a context analysis means for analyzing user utterances and extracting conversational intent.

[2234] (Claim 3)

[2235] 10. The system of claim 1, further comprising: a learning means for continuously improving the generative model using feedback data from a user.

[2236] "Example 2: Combining Emotion Engines"

[2237] (Claim 1)

[2238] A means for loading a generative AI model and initializing a user-specific model based on data including the user's conversation history;

[2239] means for converting speech input from a user into text and transmitting the text to a server;

[2240] a means for analyzing the received text by the server and obtaining emotion information using an emotion engine;

[2241] a means for generating a response based on the analysis results and the emotional information using a generative AI model;

[2242] a means for outputting the generated response to a user by voice using a voice synthesis technique;

[2243] a means for collecting feedback from users and storing the feedback and sentiment data in a database;

[2244] A system including:

[2245] (Claim 2)

[2246] 10. The system of claim 1, further comprising a context analysis means for analyzing user utterances and extracting conversational intent.

[2247] (Claim 3)

[2248] 10. The system of claim 1, further comprising: a learning means for continuously improving the generative AI model using feedback data from a user.

[2249] "Application example 2 when combining emotion engines"

[2250] (Claim 1)

[2251] A means for releasing the generative model and initializing a model for user-specific conversation generation based on organized data including the conversation history of each user;

[2252] means for converting speech input from a user into text and transmitting the text to a server;

[2253] a means for generating an appropriate response based on the received text by the server using a generative model;

[2254] means for using speech synthesis technology to audibly output the generated response to the user;

[2255] means for collecting user feedback and storing the feedback in a database for use in improving future conversation generation;

[2256] a means for using an emotion engine so that suggested responses reflect the user's emotional state;

[2257] A means for suggesting appropriate menu items to users based on their emotions and past ordering history;

[2258] A system including:

[2259] (Claim 2)

[2260] 10. The system of claim 1, further comprising a context analysis means for analyzing user utterances and extracting conversational intent.

[2261] (Claim 3)

[2262] 10. The system of claim 1, further comprising: a learning means for continuously improving the generative model using feedback data from a user. [Explanation of symbols]

[2263] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for releasing the generative model and initializing a model for user-specific conversation generation based on organized data including the conversation history of each user; means for converting speech input from a user into text and transmitting the text to a server; a means for generating an appropriate response based on the received text by the server using a generative model; means for using speech synthesis technology to audibly output the generated response to the user; means for collecting user feedback and storing the feedback in a database for use in improving future conversation generation; A system including:

2. The system of claim 1 , further comprising a context analysis means for analyzing user utterances and extracting conversational intent.

3. The system of claim 1 , further comprising: a learning means for continuously improving the generative model using feedback data from a user.

Images