System

Abstract

To provide a system.SOLUTION: A system built into an interactive robot incorporating a generative AI includes means for enabling a setting of a character of the robot in advance, means for learning a user's hobby and taste, way of thinking, and view of life by repeating a conversation, means for specifying a user's feeling state by using a feeling engine, and means for adjusting a conversation so as to change a topic on the basis of the user's learned hobby and taste, way of thinking, and view of life when determining that the user's feeling state indicates boring from a conversation with the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] There are a lack of ways to help single people and elderly people living alone feel lonely, or to help them with their worries in daily life, things they want to research, or products they want to buy. There are also no ways to understand the hobbies, tastes, ways of thinking, and outlook on life of such people, or to provide engaging conversation. Furthermore, there are a lack of ways to connect with medical institutions and local governments to respond to lonely deaths and emergencies. [Means for solving the problem]

[0005] We have developed a conversational robot incorporating generative AI and are selling it to single people and elderly people living alone. This robot can handle everything from worries to casual everyday conversations, things you want to research, and even purchasing products you want to buy. It is possible to set the robot's personality (age, conversational style) in advance, and through repeated conversations it learns the user's hobbies, preferences, way of thinking, and outlook on life. Furthermore, it provides conversations to keep users from getting bored, and based on the content of the conversations it can connect with medical institutions and local governments to respond to lonely deaths and emergencies. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] One aspect of the present invention is a conversational robot incorporating a generative AI. This robot is marketed to single people and elderly people living alone, and can handle everything from worries to everyday conversations, things to look up, and even purchasing products. Specifically, the robot has a voice recognition function and responds by analyzing voice input from the user. It can also purchase products and search for information via its internet connection.

[0029] "Example 2"

[0030] Furthermore, the robot's personality can be set in advance. Specifically, users can set the robot's age and conversation style. For example, they can set the robot to speak like a young person or to use honorific language. This allows users to enjoy conversations with the robot according to their preferences.

[0031] "Example 3"

[0032] Furthermore, through repeated conversations, the robot learns about the user's hobbies, preferences, way of thinking, and outlook on life. Specifically, the robot extracts keywords from conversations with the user and analyzes them to create a profile of the user. For example, if the user often talks about travel, the robot will learn that the user loves traveling. This allows the robot to provide conversations tailored to the user.

[0033] "Example 4"

[0034] Furthermore, the robot will contact medical institutions and local governments based on the content of the conversation to respond to cases of lonely deaths and emergencies. Specifically, the robot will collect information about the user's health and living situation and report it to medical institutions and local governments as necessary. For example, if the user complains of feeling unwell, the robot will contact a medical institution with that information, enabling early response.

[0035] The processing flow of each embodiment will be described below.

[0036] "Example 1"

[0037] Step 1: An interactive robot incorporating generative AI is launched.

[0038] Step 2: The robot uses its voice recognition function to analyze the voice input from the user.

[0039] Step 3: Based on the analysis results, the robot generates an appropriate response and outputs it to the user.

[0040] Step 4: If the user requests to purchase a product or search for information, the robot will perform the corresponding operation via its internet connection function.

[0041] "Example 2"

[0042] Step 1: The user sets the robot's personality from the settings screen.

[0043] Step 2: The user sets the robot's age and conversation style.

[0044] Step 3: Once the setup is complete, the robot will conduct a conversation based on the settings.

[0045] "Example 3"

[0046] Step 1: The robot extracts keywords from the conversation with the user.

[0047] Step 2: Analyze the extracted keywords and create a user profile.

[0048] Step 3: Based on the profile you create, the robot will provide you with a tailored conversation.

[0049] "Example 4"

[0050] Step 1: The robot collects information about the user's health and living conditions.

[0051] Step 2: If the collected information exceeds a certain threshold, the robot reports it to a medical institution or local government.

[0052] Step 3: The medical institution or local government that receives the report will take appropriate action for the user.

[0053] Example 1

[0054] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0055] In modern society, the number of single people and elderly people living alone is increasing, and these people often feel lonely and anxious. In addition, various support is needed in daily life, such as searching for information and purchasing products, but it is difficult for them to do these things alone. Furthermore, due to the lack of means to respond quickly in emergencies, lonely deaths and health problems have become serious issues.

[0056] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0057] In this invention, the server includes a means for converting voice input into text data using a voice recognition function, a means for generating appropriate responses from the text data using a generative AI model, and a means for searching for information and purchasing products using an internet connection function. This allows users to consult with the robot about their worries, engage in everyday conversations, search for information, and purchase products simply by speaking to the robot. The robot can also learn the user's preferences and way of thinking through conversations and provide more personalized responses. Furthermore, in emergencies, the robot can cooperate with medical institutions and local governments to respond quickly.

[0058] "Generative AI" is a system that uses artificial intelligence technology to generate natural language responses from text data.

[0059] An "interactive robot" is a robot that converses with users by combining technologies such as voice recognition, generative AI, and voice synthesis.

[0060] "Single people / elderly people living alone" refers to people who live alone, especially elderly people.

[0061] "Voice recognition function" is a technology that converts voice input into text data.

[0062] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on input text data.

[0063] "Internet connectivity" means the ability to access external information sources and services via the Internet.

[0064] The "voice synthesis function" is a technology that converts text data into voice and outputs it.

[0065] "Personality settings" is a function that allows you to pre-set the robot's characteristics, such as its age and conversational style.

[0066] "Hobbies and tastes" refers to the user's preferences and interests.

[0067] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[0068] "Local government" refers to a government agency responsible for local government administration.

[0069] An "emergency" refers to an urgent situation faced by a user, such as a health problem or accident.

[0070] This invention is a system that provides multifunctional support to single people and elderly people living alone using an interactive robot incorporating generative AI. This system is realized by combining technologies such as voice recognition, generative AI, internet connection, and voice synthesis.

[0071] Hardware and software used

[0072] Speech recognition function: Uses a speech recognition API (e.g., Google® Cloud Speech-to-Text API) to convert voice input into text data.

[0073] Generative AI models: Use generative AI models (e.g., OpenAI's GPT-4®) to generate appropriate responses from text data.

[0074] Internet connection functions: Use the Internet connection functions to search for information and purchase products. Specifically, use the Google Search API and Amazon API.

[0075] Speech synthesis function: Uses a speech synthesis API (e.g., Google Cloud Text-to-Speech API) to convert text data into speech and output it.

[0076] Specific operation of the system

[0077] When a user speaks to the robot, it captures the voice through its built-in microphone. It then uses a speech recognition API to convert the voice data into text data. This text data is sent to the server and input into the generative AI model. The generative AI model generates an appropriate response based on the input text data and returns the response to the server.

[0078] The server receives the response from the generative AI model and, if necessary, uses its internet connection to retrieve additional information. For example, if a user asks, "What restaurants are nearby?", the server uses the Google Search API to retrieve information about nearby restaurants. The retrieved information is then combined with the response from the generative AI model to generate the final response.

[0079] The final response is sent from the server to the device, which then uses a speech synthesis API to convert the text data into speech, which is then spoken to the user.

[0080] Specific examples

[0081] Example 1: Everyday conversation

[0082] User: "How's the weather today?"

[0083] Robot: "It's sunny today and the temperature is 25 degrees."

[0084] Example 2: Information search

[0085] User: "What restaurants are nearby?"

[0086] Robot: "There's an Italian restaurant nearby. It has a 4.5 rating."

[0087] Example 3: Purchasing a product

[0088] User: "I want to buy a new smartphone."

[0089] Robot: "The latest smartphone is on sale in the online store. Would you like to buy it?"

[0090] Prompt Sentence Examples

[0091] "When a user speaks to the robot, it uses speech recognition to convert the speech into text, which is then fed into a generative AI model to generate a response. For example, if a user asks, 'What's the weather like today?' the robot will respond, 'It's sunny today, and the temperature is 25 degrees.'"

[0092] In this way, the system of the present invention can meet a variety of user needs by combining voice recognition, generation AI, internet connection, and voice synthesis.

[0093] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0094] Step 1:

[0095] The user provides voice input.

[0096] The user speaks to the robot, for example, asking, "What's the weather like today?" The input is the user's voice, and the output is the audio data captured by the robot's microphone.

[0097] Step 2:

[0098] The device converts the speech to text.

[0099] The device captures the user's voice through the built-in microphone. Then, it uses a speech recognition API to convert the voice data into text data. The input is the voice data, and the output is the text data "What's the weather like today?"

[0100] Step 3:

[0101] The terminal transmits the text data to the server.

[0102] The terminal sends the converted text data to the server. The input is the text data, and the output is the text data sent to the server.

[0103] Step 4:

[0104] The server sends the text data to the generative AI model.

[0105] The server inputs the received text data into the generative AI model. The input is the text data "What's the weather like today?", and the output is the prompt input into the generative AI model.

[0106] Step 5:

[0107] The server receives the response from the generative AI model.

[0108] The generative AI model generates an appropriate response based on the input text data. The server receives this response. The input is the prompt entered into the generative AI model, and the output is the response text: "Today is sunny and the temperature is 25 degrees."

[0109] Step 6:

[0110] The server performs internet searches and purchases as needed.

[0111] When a user requests information search or product purchase, the server uses its Internet connection to obtain the necessary information. For example, it uses the Google Search API to obtain weather information. The input is text data based on the user's request, and the output is the obtained information.

[0112] Step 7:

[0113] The server sends a final response to the terminal.

[0114] The server compiles the responses from the generative AI model and the information obtained from the internet to generate a final response, which is then sent to the terminal. The input is the response text from the generative AI model and the obtained information, and the output is the final response text sent to the terminal.

[0115] Step 8:

[0116] The terminal responds to the user verbally.

[0117] The device converts the final response received from the server into speech using a speech synthesis API. It responds to the user via voice. The input is the final response text, and the output is a voice response to the user. For example, it might respond, "Today is sunny, and the temperature is 25 degrees."

[0118] (Application example 1)

[0119] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0120] Single people and elderly people living alone often feel lonely and anxious in their daily lives, and require support, especially when it comes to ordering meals and managing their health. However, there are only a limited number of effective systems to address these needs. There is also a lack of systems that can provide appropriate suggestions based on the user's hobbies, tastes, and health status. This creates a challenge for users, making it difficult for them to live their lives with peace of mind.

[0121] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0122] In this invention, the server is an interactive robot incorporating a generative AI, a system for selling to single people and elderly people living alone, and includes: means for the robot to respond to everything from worries to casual everyday conversations, things you want to research, and purchasing products you want; means for setting the robot's personality in advance (age, conversational style); means for learning the user's hobbies, preferences, way of thinking, and outlook on life through repeated conversations; means for analyzing voice input and ordering food; means for connecting to a food delivery service via the Internet and completing the order; and means for storing the user's food preferences and allergy information and suggesting appropriate menus. This helps reduce the user's sense of loneliness and anxiety, allowing them to live their lives with peace of mind.

[0123] "Generative AI" refers to algorithms and models that use artificial intelligence techniques to generate new data and information.

[0124] An "interactive robot" is a robot that can converse with humans using voice recognition and natural language processing technology.

[0125] A "single person" is someone who lives alone.

[0126] "Elderly people living alone" refers to elderly people who live alone.

[0127] "Personality setting" refers to setting a robot's characteristics, such as its age and how it speaks, in advance.

[0128] Hobbies and interests are the activities and things that an individual likes.

[0129] "Audio input" refers to obtaining audio as digital data through a device such as a microphone.

[0130] "Ordering food" refers to the act of requesting food from a restaurant or food delivery service.

[0131] "Internet connection" means access to the Internet through a network.

[0132] "Food delivery service" means a service that delivers ordered meals to a specified location.

[0133] "Allergy information" refers to information that indicates an allergic reaction to a specific food ingredient or substance.

[0134] "Menu suggestion" refers to recommending an appropriate meal menu based on the user's preferences and health condition.

[0135] The system for implementing this invention uses an interactive robot incorporating generative AI. The system is marketed to single people and elderly people living alone, and can handle everything from worries to everyday conversations, things you want to research, and even purchasing products you want to buy. Specific embodiments of the system are described below.

[0136] Hardware and Software Configuration

[0137] The system uses the following hardware and software:

[0138] Hardware: Microphone (for voice input), internet-enabled robot

[0139] Software: Python, SpeechRecognition library, OpenAI API, Requests library

[0140] Data processing and calculation

[0141] 1. Acquiring voice input:

[0142] The user provides voice input through a microphone.

[0143] Use the speech recognition library (SpeechRecognition) to convert speech into text data.

[0144] 2. Use of generative AI models:

[0145] The converted text data is sent as a prompt to the generative AI model (OpenAI API).

[0146] The AI ​​model generates an appropriate response based on the prompt.

[0147] 3. Food delivery orders:

[0148] Connect to the API of a food delivery service based on the user's order.

[0149] Use the Requests library to send the order data to the food delivery service and complete the order.

[0150] 4. User information storage and suggestions:

[0151] The database stores the user's dietary preferences and allergy information.

[0152] The next time you order, the system will suggest an appropriate menu item based on the stored information.

[0153] Specific examples

[0154] For example, if a user says, "I want to order a pizza," the system operates as follows:

[0155] 1. Acquiring voice input:

[0156] The user speaks into the microphone, "I'd like to order a pizza."

[0157] The speech recognition library converts this speech into text, generating the text data "I would like to order a pizza."

[0158] 2. Use of generative AI models:

[0159] The generated text data is sent to the OpenAI API as a prompt.

[0160] Example prompt: "The meal the user would like to order is: Pizza."

[0161] The AI ​​model generates the response, "What kind of pizza would you like?"

[0162] 3. Food delivery orders:

[0163] The user responds, "Margherita pizza."

[0164] The system sends this information to the food delivery service's API to complete the order.

[0165] 4. User information storage and suggestions:

[0166] The database stores that the user likes Margherita pizza.

[0167] From the next time onwards, the user will be suggested a Margherita pizza the next time they order pizza.

[0168] In this way, the system can help single people and elderly people living alone to easily order meals, improving the quality of their daily lives.

[0169] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0170] Step 1:

[0171] The user provides voice input through a microphone. When the user says, "I'd like to order a pizza," the microphone captures the voice. The input is the voice data, and the output is the captured voice data.

[0172] Step 2:

[0173] The device uses a speech recognition library (SpeechRecognition) to convert the captured voice data into text data. Specifically, the speech recognition library analyzes the voice data and generates the text data "I would like to order a pizza." The input is voice data, and the output is text data.

[0174] Step 3:

[0175] The device sends the generated text data as a prompt to the generative AI model (OpenAI API). Specifically, the text data "I would like to order pizza" is sent as a prompt to the AI ​​model, which generates an appropriate response. The input is text data, and the output is the response text generated by the AI ​​model.

[0176] Step 4:

[0177] The server receives the response from the generative AI model and responds to the user by voice, asking, "What kind of pizza would you like?" Specifically, the generated response text is converted into voice data using speech synthesis technology and transmitted to the user through a speaker. The input is the response text, and the output is voice data.

[0178] Step 5:

[0179] The user again inputs voice through the microphone and answers "Margherita pizza." The microphone captures the voice and sends it to the device as voice data. The input is voice data, and the output is the captured voice data.

[0180] Step 6:

[0181] The device again uses the speech recognition library to convert the captured voice data into text data. Specifically, the speech recognition library analyzes the voice data and generates the text data "Margherita pizza." The input is voice data, and the output is text data.

[0182] Step 7:

[0183] The server sends the text data "Margherita Pizza" to the food delivery service's API to complete the order. Specifically, it uses the Requests library to send the text data as a POST request to the food delivery service's API endpoint. The input is the text data, and the output is a confirmation of the order completion from the food delivery service.

[0184] Step 8:

[0185] The server stores the user's food preferences and allergy information in a database. Specifically, the user's preference for "Margherita pizza" is saved in the database and used for future orders. The input is the user's preferences and allergy information, and the output is the information saved in the database.

[0186] Step 9:

[0187] From the next time onwards, when the user orders a meal again, the server retrieves the user's preferences and allergy information from the database and suggests an appropriate menu. Specifically, based on the information retrieved from the database, the server suggests to the user, "How about a Margherita pizza?" The input is the information retrieved from the database, and the output is the suggested text.

[0188] Example 2

[0189] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0190] In modern society, the number of single people and elderly people living alone is increasing, and these people often feel lonely. Furthermore, they need various support in their daily lives, such as helping with worries, researching things they want to know, and purchasing products they want to buy, but it is difficult to solve these problems on their own. Furthermore, they are required to learn the user's hobbies, tastes, and ways of thinking, and to provide continuously interesting conversations. Conversational robots are effective in solving these problems, but current systems are not able to adequately address them.

[0191] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0192] In this invention, the server includes means for a user to set the robot's personality using a terminal, means for the terminal to send the setting data to the server, means for the server to send a prompt sentence to the generative AI model, means for the generative AI model to generate a response, means for the server to send the generated response to the robot, and means for the robot to converse with the user, thereby enabling the user to enjoy a conversation with the robot according to their preferences.

[0193] A "generative AI model" is a type of artificial intelligence that generates text data using natural language processing technology.

[0194] A "prompt sentence" is an input sentence that prompts a generative AI model to generate a specific response.

[0195] An "interactive robot" is a robot that can converse with a user through voice or text.

[0196] A "terminal" is a device operated by a user, and includes smartphones, personal computers, etc.

[0197] A "server" is a computer system that processes and stores data over a network.

[0198] "Personality settings" is a feature that allows users to customize the robot's age and conversation style.

[0199] A "response" is text data that a generative AI model generates based on a prompt sentence.

[0200] A "user" is a person who operates an interactive robot and has a conversation with it.

[0201] "Setting data" is information about the robot's personality settings that the user inputs through the terminal.

[0202] A "conversation" is a communication exchange between a user and an interactive robot.

[0203] This invention is a system that uses an interactive robot incorporating a generative AI model to help single people and elderly people living alone reduce their feelings of loneliness and receive support in their daily lives. Specific embodiments of this system are described below.

[0204] Hardware and software used

[0205] Hardware: Robot body, server, user terminal (PC, smartphone, etc.)

[0206] Software: Generative AI models (e.g., models using natural language processing techniques), robot control software, user interface (UI) software

[0207] Data processing and calculation

[0208] 1. The user sets the robot's personality using the device

[0209] Users use the device's UI to set the robot's age and how it will converse.

[0210] Example: A user opens an app on their smartphone and selects "Age: Young" and "Conversational style: Casual" on the settings screen.

[0211] 2. The device sends the configuration data to the server.

[0212] The terminal transmits the setting data entered by the user to the server.

[0213] Example: The device packages configuration data in JSON format and sends it to the server using the HTTPS protocol.

[0214] 3. The server sends a prompt to the generative AI model

[0215] The server sends a prompt to the generative AI model based on the received configuration data.

[0216] Example: The server generates a prompt saying, "Generate a response from a robot that can have casual conversations like a young person," and sends it to the API endpoint of the generative AI model.

[0217] 4. The generative AI model generates a response

[0218] The generative AI model generates the robot's response based on the prompt sentence.

[0219] Example: A generative AI model parses a prompt and generates the response "Hey there! What are you doing today?"

[0220] 5. The server generates a response and sends it to the robot

[0221] The server sends the response received from the generative AI model to the robot.

[0222] Example: The server sends the response data to the robot control software, which outputs it as sound from the robot's speaker.

[0223] 6. The robot will converse with the user

[0224] The robot will then converse with the user based on the received response.

[0225] Example: A robot speaks to the user, "Hey there! What are you doing today?" When the user responds, the robot sends the data back to the server and the process repeats to generate the next response.

[0226] Prompt Sentence Examples

[0227] "Generate responses from a robot that can have casual conversations like a young person."

[0228] "Generate a polite robotic response using honorific language."

[0229] This system allows users to enjoy conversations with robots according to their preferences, reducing feelings of loneliness and providing support for daily life.

[0230] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0231] Step 1:

[0232] The user uses the terminal to set the robot's personality.

[0233] Input: Configuration data that users enter through the device's UI (e.g., age, speech patterns).

[0234] Specific actions: The user opens the app on their smartphone and selects "Age: Young" and "Conversation style: Casual" on the settings screen.

[0235] Output: The configuration data is saved to the device.

[0236] Step 2:

[0237] The terminal sends the setting data to the server.

[0238] Input: Configuration data entered by the user.

[0239] What happens: The device packages the configuration data in JSON format and sends it to the server using the HTTPS protocol.

[0240] Output: The configuration data is sent to the server.

[0241] Step 3:

[0242] The server sends a prompt to the generative AI model.

[0243] Input: The configuration data received by the server.

[0244] Specific operation: The server generates a prompt sentence, "Please generate a response from a robot that can have casual conversation like a young person," and sends it to the API endpoint of the generative AI model.

[0245] Output: The prompt is sent to the generative AI model.

[0246] Step 4:

[0247] A generative AI model generates a response.

[0248] Input: The prompt received by the generative AI model.

[0249] What it does: The generative AI model analyzes the prompt and generates the response, "Hey there! What are you doing today?"

[0250] Output: The generated response is sent back to the server.

[0251] Step 5:

[0252] The server generates a response and sends it to the robot.

[0253] Input: The response returned by the generative AI model.

[0254] Specific operation: The server sends the response data to the robot control software, which outputs it as voice from the robot's speaker.

[0255] Output: The response data is sent to the robot.

[0256] Step 6:

[0257] The robot converses with the user.

[0258] Input: The response data received by the robot.

[0259] What it does: The robot asks the user, "Hey! What are you doing today?" When the user responds, the robot sends the data back to the server and the process repeats, generating the next response.

[0260] Output: The conversation with the user continues.

[0261] (Application example 2)

[0262] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0263] In modern society, the number of single people and elderly people living alone is increasing, and supporting their loneliness and daily life is becoming an important issue. Another problem is the lack of people to talk to to alleviate the worries and anxieties they experience on a daily basis. Furthermore, there is a need to realize more friendly and timeless communication by providing conversations tailored to the user's preferences.

[0264] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server is an interactive robot incorporating a generative AI, and is a system for selling to single people and elderly people living alone. The system includes: means for the robot to respond to everything from worries to casual everyday conversations, things to research, and purchasing products; means for enabling the robot's personality to be set in advance (age, conversational style); means for learning the user's hobbies, tastes, way of thinking, and outlook on life through repeated conversations; means for the user to set the robot's personality using a smart device; means for generating natural conversations using a generative AI model based on the set personality; and means for the robot to speak the generated conversation content to the user. This allows for dialogue tailored to the user's preferences, reducing feelings of loneliness and supporting daily life.

[0265] "Generative AI" is a system that uses artificial intelligence technology to process natural language and generate dialogue with users.

[0266] An "interactive robot" is a robot that can communicate with users through voice and text.

[0267] "Single person" refers to someone who lives alone.

[0268] "Elderly people living alone" refers to elderly people who live alone.

[0269] A "system" is a structure in which multiple elements are combined to achieve a specific function.

[0270] "Worries" refer to problems and anxieties that users experience in their daily lives.

[0271] "Daily conversation" refers to general conversation in everyday life.

[0272] "What you want to know" refers to the information or questions that users want to know.

[0273] "Products you want to buy" refers to items that users are considering purchasing.

[0274] "Personality settings" refers to the user pre-setting the robot's age and conversation style.

[0275] "Hobbies and tastes" refers to the user's interests and preferences.

[0276] "Thoughts" refers to the user's thoughts and opinions.

[0277] "Life philosophy" refers to the user's perspective and values ​​on life.

[0278] "Smart devices" refer to electronic devices that can connect to the Internet, such as smartphones and tablets.

[0279] "Generative AI model" refers to the algorithms and data models used to realize generative AI.

[0280] "Natural conversation" refers to conversations that are generated in a manner that is close to dialogue between humans.

[0281] The system for implementing this invention uses an interactive robot incorporating generative AI. The system is marketed to single people and elderly people living alone, and can handle everything from worries to everyday conversations, research needs, and product purchases. A specific embodiment of this system is described below.

[0282] System configuration

[0283] The system consists of the following main components:

[0284] 1. Conversational robot: A robot that communicates with users through voice and text.

[0285] 2. Smart Device: A device such as a smartphone or tablet that allows the user to set the robot's personality.

[0286] 3. Generative AI models: Algorithms and data models that perform natural language processing and generate user interactions.

[0287] Program processing

[0288] The server implements the system using the following means:

[0289] 1. Personality setting method: The user uses a smart device to set the robot's age and conversation style. For example, if the user sets the robot to "speak casually like a young person," the prompt text will be as follows:

[0290] Speak casually, like you're a young person.

[0291] 2. Conversation generation: A generative AI model is used based on the personality settings to generate natural conversations. The generative AI model used is OpenAI's GPT-3 (registered trademark), among others.

[0292] 3. Conversation output means: The robot speaks the generated conversation content to the user, allowing the user to enjoy interacting with the robot.

[0293] Specific examples

[0294] For example, when a user sets a robot's personality using a smartphone, the procedure is as follows:

[0295] 1. The user opens the smartphone application and sets the robot's age and conversation style.

[0296] 2. The configuration information is sent to the server and a prompt is generated.

[0297] 3. Based on the prompt, the generative AI model generates natural conversation.

[0298] 4. The generated conversation content is sent to the robot, which then speaks to the user.

[0299] In this way, the system can provide interactions tailored to the user's preferences, reducing feelings of loneliness and supporting daily life.

[0300] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0301] Step 1:

[0302] The user configures the robot's personality using a smart device. Specifically, the user opens an application on their smartphone or tablet and configures the robot's age and conversation style. The input includes the user's selection of age and conversation style. The output is the configuration information.

[0303] Step 2:

[0304] The terminal sends the user's setting information to the server. Specifically, the setting information is sent from the smart device to the server. The input includes the user's age and conversation style. The server receives the setting information as output.

[0305] Step 3:

[0306] The server generates a prompt sentence based on the configuration information it receives. Specifically, it analyzes the configuration information and creates a prompt sentence suitable for the generative AI model. The input includes the user's configuration information. The output is the generated prompt sentence. For example, a generated prompt sentence might be, "Please speak casually, like a young person."

[0307] Step 4:

[0308] The server uses a generative AI model to generate natural conversations. Specifically, the generated prompt sentence is input into a generative AI model (e.g., OpenAI's GPT-3) to generate the conversation content. The input includes the prompt sentence. The output is the generated conversation content.

[0309] Step 5:

[0310] The server sends the generated conversation content to the robot. Specifically, the server sends the generated conversation content to the interactive robot, and the robot prepares to speak the content to the user. The generated conversation content is included as input. The robot receives the conversation content as output.

[0311] Step 6:

[0312] The robot speaks the generated conversation content to the user. Specifically, the robot uses speech synthesis technology to convey the generated conversation content to the user by voice. The generated conversation content is included as input. The output allows the user to enjoy a dialogue with the robot.

[0313] Example 3

[0314] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0315] There is a need to reduce loneliness among single people and elderly people living alone and to provide support in their daily lives. It is also important to monitor the user's health and living situation and respond quickly to emergencies. Conventional interactive robots have had difficulty fully understanding the user's hobbies and health and responding appropriately. Another challenge is maintaining a conversation so that the user does not get bored.

[0316] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0317] In this invention, the server includes a means for collecting conversations with the user as voice input, a means for converting the collected voice data into text data, and a means for extracting keywords from the converted text data, thereby enabling the server to understand the user's hobbies and health condition and provide appropriate support.

[0318] "Generative AI" is a system that uses artificial intelligence technology to generate data such as text and voice.

[0319] An "interactive robot" is a robot that can converse with users through voice or text.

[0320] "Single people / elderly people living alone" refers to people who live alone, and specifically includes elderly people.

[0321] "Personality setting" refers to setting a robot's characteristics, such as its age and how it speaks, in advance.

[0322] "Hobbies and interests" refers to the activities and areas of interest that users prefer.

[0323] A "profile" is data that compiles information such as a user's hobbies, thoughts, and health status.

[0324] "Voice input" refers to collecting a user's speech as voice data.

[0325] "Text data" refers to data obtained by converting voice data into character information.

[0326] "Keywords" are important words and phrases extracted from conversations.

[0327] "Generative AI model" refers to an algorithm or system that uses artificial intelligence technology to generate text or speech.

[0328] "Audio output" refers to the presentation of generated text or information to the user as audio.

[0329] "Health status" refers to information about the user's physical condition and medical care.

[0330] "Living situation" refers to information about the user's daily life.

[0331] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[0332] "Local government" refers to a local administrative body that plays a role in supporting the lives of residents.

[0333] This invention is a system that uses an interactive robot incorporating a generative AI to help single people and elderly people living alone reduce feelings of loneliness and provide support for their daily lives. Specific embodiments of this system are described below.

[0334] Hardware and software used

[0335] Hardware: Interactive robots (e.g., general robotic platforms)

[0336] Software: Natural language processing engines (e.g., general natural language processing APIs), generative AI models (e.g., general generative AI models)

[0337] Explanation of program processing

[0338] Conversation data collection

[0339] The terminal (interactive robot) collects conversations with the user as voice input. Specifically, when the user speaks to the robot, the robot captures the voice with a microphone and saves it as voice data.

[0340] Keyword extraction

[0341] The device uses speech recognition software to convert the collected voice data into text data. The server then uses a natural language processing engine to extract keywords from the converted text data. For example, if the voice data is converted into text such as "I want to go on a trip recently," keywords such as "travel" and "want to go" are extracted.

[0342] Creating a profile

[0343] The server analyzes the extracted keywords and creates a user profile. For example, if the keyword "travel" appears frequently, the server records the user as "travel lover" in the user's profile.

[0344] Conversational Offerings

[0345] The server uses the generative AI model to generate a conversation based on the user's profile. The device then provides the generated conversation to the user as voice output. For example, the server generates a question such as, "Is there anywhere you'd like to travel to recently?", and the device then verbally replies the question to the user.

[0346] Health monitoring

[0347] The device collects information about the user's health and lifestyle during the conversation. The server analyzes the collected information and checks for any abnormalities. For example, if a user says, "I haven't been feeling well lately," the device sends that information to the server, which then analyzes it.

[0348] Emergency response

[0349] The server will report the results of the health status analysis to medical institutions or local governments as necessary. For example, if the server detects that the user is in poor health, it will automatically contact a medical institution by email or phone.

[0350] Prompt Sentence Examples

[0351] Prompt: "What questions should be generated when a user talks about a recent trip?"

[0352] Example output from a generative AI model: "Is there anywhere you'd like to travel to these days?"

[0353] This system enables the robot to understand the user's hobbies and health condition and take appropriate measures. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0354] Step 1:

[0355] The device collects conversations with users as voice input. Specifically, when a user speaks to the robot, the robot captures the voice with a microphone and saves it as voice data.

[0356] Input: User speech

[0357] Output: Audio data

[0358] Step 2:

[0359] The device uses voice recognition software to convert the collected voice data into text data. For example, the voice recognition software analyzes the voice data and converts it into text such as "I want to go on a trip these days."

[0360] Input: Audio data

[0361] Output: Text data

[0362] Step 3:

[0363] The server then uses a natural language processing engine to extract keywords from the converted text data. Specifically, the natural language processing engine analyzes the text data and extracts keywords such as "travel" and "want to go."

[0364] Input: Text data

[0365] Output: Keywords

[0366] Step 4:

[0367] The server analyzes the extracted keywords and creates a user profile. For example, if the keyword "travel" appears frequently, the server records the user as "travel lover" in the user's profile.

[0368] Input: keyword

[0369] Output: User profile

[0370] Step 5:

[0371] The server uses a generative AI model to generate a conversation based on the user's profile. Specifically, the generative AI model generates a question based on the profile information, such as "Is there anywhere you'd like to travel to recently?"

[0372] Input: User Profile

[0373] Output: Generated conversation text

[0374] Step 6:

[0375] The device provides the generated conversation to the user as a voice output. Specifically, the device converts the generated text into voice using speech synthesis software and conveys the voice to the user.

[0376] Input: Generated conversation text

[0377] Output: Audio output

[0378] Step 7:

[0379] The device collects information about the user's health and lifestyle during conversations. For example, if the user says, "I haven't been feeling well lately," the device collects that information as voice data.

[0380] Input: User speech

[0381] Output: Audio data about health status

[0382] Step 8:

[0383] The server converts the collected voice data about health conditions into text data for analysis. Specifically, voice recognition software converts the voice data into text, and a natural language processing engine analyzes the text.

[0384] Input: Audio data about health conditions

[0385] Output: Text data about health conditions

[0386] Step 9:

[0387] The server will report the results of the health status analysis to medical institutions or local governments as necessary. Specifically, if the server detects an abnormality, it will automatically contact the medical institution by email or phone.

[0388] Input: Text data about health conditions

[0389] Output: Report to medical institutions and local authorities

[0390] (Application example 3)

[0391] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0392] In modern society, the number of single people and elderly people living alone is increasing, and their feelings of loneliness and how to respond in emergencies are becoming major issues. There is also a need to understand their hobbies and health conditions and provide appropriate support. However, conventional interactive robots have had difficulty fully learning about users' hobbies and health conditions and responding appropriately in emergencies.

[0393] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server is an interactive robot incorporating a generative AI, and is a system for selling to single people and elderly people living alone. The system includes: means for the robot to respond to everything from worries to casual everyday conversations, things to research, and purchasing desired products; means for enabling the robot's personality to be set in advance (age, conversational style); means for learning the user's hobbies, tastes, way of thinking, and outlook on life through repeated conversations; means for collecting information on the user's health condition and living situation and reporting it to medical institutions or local governments as necessary; and means for learning the user's hobbies and health condition through daily conversations and contacting medical institutions or local governments in case of an emergency. This makes it possible to understand the user's hobbies and health condition and provide appropriate support.

[0394] "Generative AI" is a system that uses artificial intelligence technology to generate new information and content from data.

[0395] An "interactive robot" is a robot that can have a natural conversation with a user.

[0396] "Single person" refers to someone who lives alone.

[0397] "Elderly people living alone" refers to elderly people who live alone.

[0398] A "system" is a mechanism in which multiple elements are combined to achieve a specific function.

[0399] "Problems" refer to problems or difficult situations that users face.

[0400] "Daily conversation" refers to general conversation in everyday life.

[0401] "What you want to find out" refers to the information or data that the user wants to know.

[0402] The "item you want to buy" refers to an item that you want to purchase.

[0403] "Personality setting" refers to setting a robot's characteristics, such as its age and how it speaks, in advance.

[0404] "Hobbies and tastes" refers to the activities and interests that users prefer.

[0405] "Mindset" refers to the user's thinking tendencies and values.

[0406] "Life perspective" refers to the user's perspective and feelings about life.

[0407] "Health Status" refers to the state of a user's physical and mental health.

[0408] "Living conditions" refers to the environment and circumstances of the user's daily life.

[0409] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[0410] "Local government" refers to a local public body responsible for local administration.

[0411] "Emergency" means a situation in which the User requires immediate attention.

[0412] "Reporting" refers to conveying collected information to relevant authorities.

[0413] The system for implementing this invention uses a conversational robot incorporating generative AI. The system is marketed to single people and elderly people living alone, and can handle everything from worries to everyday conversations, things they want to research, and even purchasing products they want to buy. It also allows the robot's personality (age, conversational style) to be set in advance, and through repeated conversations, it learns the user's hobbies, preferences, way of thinking, and outlook on life.

[0414] Program processing explanation

[0415] Hardware

[0416] Smartphone: A device that users use on a daily basis.

[0417] Server: Processes data and runs generative AI models.

[0418] software

[0419] OpenAI API: Uses generative AI models to analyze conversation data and create user profiles.

[0420] requests library: Sends HTTP requests to contact medical institutions and local authorities in emergencies.

[0421] Data processing and calculation

[0422] 1. Collection of conversation data: Collect users' everyday conversations via smartphones.

[0423] 2. Profile creation: Using the OpenAI API, the user's hobbies and health status are extracted from the collected conversation data to create a profile.

[0424] 3. Emergency response: If a user complains of feeling unwell, the requests library will be used to automatically contact medical institutions and local authorities.

[0425] Specific examples

[0426] For example, if a user says, "I'd like to travel recently, but I haven't been feeling well," the system collects this conversation and uses the OpenAI API to input the following prompt sentence into the generative AI model:

[0427] Prompt Sentence Examples

[0428] Extract the user's hobbies and health status from the following conversation:

[0429] I've been wanting to go on a trip lately, but I haven't been feeling well.

[0430] Based on this prompt, the generative AI model learns that the user is a travel enthusiast and is currently unwell. It then uses the requests library to send a message to a medical institution saying, "The user is feeling unwell. Please take immediate action."

[0431] In this way, the system can understand the user's hobbies and health status and provide appropriate support.

[0432] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0433] Step 1:

[0434] A user has everyday conversations through a smartphone. The input is the user's voice data, which the smartphone converts into text data. The output is text data.

[0435] Step 2:

[0436] The smartphone sends text data to the server. The input is the text data, and the output is the text data sent to the server.

[0437] Step 3:

[0438] The server uses the OpenAI API to extract the user's hobbies and health status from the text data. The input is the text data and a prompt, which is "Please extract the user's hobbies and health status from the following conversation: [text data]." The output is the user's profile information.

[0439] Step 4:

[0440] The server stores the user's profile information in a database. The input is the user's profile information, and the output is the profile information stored in the database.

[0441] Step 5:

[0442] The server analyzes the user's profile information and determines whether emergency response is required. The input is the profile information, and the output is a flag indicating whether emergency response is required.

[0443] Step 6:

[0444] If an emergency response is deemed necessary, the server contacts medical institutions and local authorities using the requests library. The input is the emergency response flag and contact information, and the output is the HTTP request sent.

[0445] Step 7:

[0446] The server records the contact result in a database and notifies the user. The input is the result of the HTTP request, and the output is the result recorded in the database and a notification to the user.

[0447] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0448] "Example 1"

[0449] One embodiment of the present invention is a system that combines an emotion engine with a conversational robot incorporating generative AI. This system recognizes emotions from the user's tone of voice, speaking style, and choice of words, and adjusts the robot's response accordingly. For example, if the user is sad, the robot will choose comforting words, and if the user is happy, it will choose words that share that joy.

[0450] "Example 2"

[0451] The emotion engine also connects with medical institutions and local governments based on the user's emotional state. For example, if the user continues to show sadness for a long period of time, the robot will report that information to medical institutions and local governments. This makes it possible to properly manage the user's mental health and provide appropriate support as needed.

[0452] "Example 3"

[0453] Furthermore, the emotion engine has the ability to adjust the conversation so that the user does not get bored. For example, if the robot senses that the user is bored, it will change the topic or tell a joke to keep the user interested. This prevents conversations with the robot from becoming monotonous and encourages users to continue using the robot for a long period of time.

[0454] The processing flow of each embodiment will be described below.

[0455] "Example 1"

[0456] Step 1: The user talks to the robot.

[0457] Step 2: The emotion engine recognizes the user's emotions from their tone of voice, speaking style, and choice of words.

[0458] Step 3: Adjust the robot's response depending on the emotion recognized by the emotion engine.

[0459] "Example 2"

[0460] Step 1: The emotion engine monitors the user's emotional state over time.

[0461] Step 2: If the user continues to show sad emotions for a long period of time, the emotion engine will report this information to medical institutions and local authorities.

[0462] Step 3: Medical institutions and local authorities receive the report and provide appropriate assistance as needed.

[0463] "Example 3"

[0464] Step 1: The emotion engine monitors the user's emotional state.

[0465] Step 2: If the user feels bored, the emotion engine instructs the robot to change the topic or tell a joke.

[0466] Step 3: The robot follows instructions from the emotion engine and engages in a conversation that engages the user.

[0467] Example 1

[0468] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0469] In modern society, the number of single people and elderly people living alone is increasing, and the loneliness and inconveniences they face in their daily lives are becoming serious problems. In particular, they face challenges such as a lack of everyday conversation skills, not having anyone to talk to about their worries, and difficulty finding necessary information or purchasing products. There is also a need to understand the emotions of these people and respond appropriately, but existing technologies are not sufficient. Furthermore, rapid response in emergencies and cooperation with medical institutions and local governments are also important challenges.

[0470] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0471] In this invention, the server includes a means for analyzing voice input from a user using a voice recognition function, a means for recognizing emotions from the user's tone of voice, speaking style, and vocabulary using an emotion engine, and a means for generating a response based on the user's emotions and input using a generative AI model. This reduces the user's sense of loneliness and eliminates inconveniences in daily life. Providing appropriate responses according to the user's emotions enables more intimate conversations. Furthermore, by including a means for searching for information and purchasing products via an internet connection function, user convenience can be improved. Additionally, by including a means for coordinating with medical institutions and local governments, it becomes possible to respond quickly to emergencies.

[0472] "Generative AI" is a system that uses artificial intelligence technology to generate data such as text and voice.

[0473] An "interactive robot" is a robot that has the ability to converse with humans through voice and text.

[0474] "Single person" refers to someone who lives alone.

[0475] "Elderly people living alone" refers to elderly people who live alone.

[0476] "Voice recognition function" is a technology that converts voice data into text data.

[0477] An "emotion engine" is a technology that analyzes a user's emotions from voice and text.

[0478] A "generative AI model" is an algorithm that uses artificial intelligence techniques to generate responses based on input data.

[0479] "Speech synthesis function" is a technology that converts text data into voice data.

[0480] "Internet connection function" refers to the function of accessing external information and services via the Internet.

[0481] "Information search" is the act of finding necessary information using the Internet or databases.

[0482] "Purchasing a product" refers to the act of ordering and purchasing a product via the Internet.

[0483] "Medical institutions" are facilities that provide medical services, such as hospitals and clinics.

[0484] A "local government" is a local public body responsible for administering a region.

[0485] "Emergency" refers to an urgent or critical situation faced by a User.

[0486] This invention provides a conversational robot incorporating generative AI, and is a system that targets single people and elderly people living alone, providing advice on worries, daily conversation, information searches, product purchases, etc. Furthermore, by combining it with an emotion engine, it can respond according to the user's emotions.

[0487] Hardware and software used

[0488] The terminal (robot) uses the following hardware and software:

[0489] Microphone: A device for capturing audio input

[0490] Speaker: A device for audio output.

[0491] Internet connection module: A device for connecting to an external network

[0492] The server uses the following software:

[0493] Speech recognition software (e.g., Google Cloud Speech-to-Text): converts voice data into text data

[0494] Emotion engine (e.g., IBM Watson® Tone Analyzer): Analyzes emotions from voice and text

[0495] Generative AI models (e.g., OpenAI GPT-4): generate responses based on input data

[0496] Speech synthesis software (e.g., Amazon Polly): converts text data into speech data

[0497] Program processing

[0498] The user speaks to the robot, for example, saying, "I'm very tired today."

[0499] The terminal (robot) captures the user's voice using a built-in microphone and transmits the voice data to the server in real time.

[0500] The server passes the received voice data to speech recognition software, which converts the voice data into text data, such as "I'm very tired today."

[0501] The server passes the converted text and voice data to an emotion engine, which analyzes the user's emotions based on their tone of voice, speaking style, and choice of words. For example, it can recognize that the user is tired.

[0502] The server passes the emotion recognition results and text data to a generative AI model, which generates an appropriate response based on the user's emotions and input. For example, the response might be, "You've had a tough day. Take a break and relax."

[0503] The server passes the generated text response to speech synthesis software, which converts the text data into speech data.

[0504] The device (robot) plays the generated voice data back to the user through a speaker, for example, saying, "Today was tough. Let's take a break and relax."

[0505] Examples of specific examples and prompts

[0506] For example, if a user says, "I'm so tired today," the robot will use its emotion engine to recognize that the user is tired, and the generative AI model will generate a response such as, "You've had a hard day today. Take a break and relax."

[0507] Example prompt sentence:

[0508] "Generate comforting words when the user is tired. The user's input is 'I'm very tired today.'"

[0509] In this way, the system combines voice recognition, emotion recognition, generative AI models, speech synthesis, and internet connectivity to provide appropriate responses to users.

[0510] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0511] Step 1:

[0512] The user speaks to the robot. For example, they say, "I'm very tired today." The terminal (robot) captures the user's voice using a built-in microphone. The input is the user's voice data, and the output is the captured voice data. Specifically, the microphone converts the voice into a digital signal and sends that data to the server.

[0513] Step 2:

[0514] The server passes the received voice data to speech recognition software. The speech recognition software (e.g., Google Cloud Speech-to-Text) converts the voice data into text data. The input is voice data, and the output is text data. Specifically, the speech recognition algorithm analyzes the voice waveform and generates corresponding text. For example, the generated text might say, "I'm very tired today."

[0515] Step 3:

[0516] The server passes the converted text and voice data to an emotion engine. The emotion engine (e.g., IBM Watson Tone Analyzer) analyzes emotions from the user's tone of voice, speaking style, and choice of words. The input is text and voice data, and the output is emotion data. Specifically, the emotion engine analyzes the characteristics of the voice and text and recognizes that the user is tired.

[0517] Step 4:

[0518] The server passes the emotion recognition results and text data to a generative AI model. The generative AI model (e.g., OpenAI GPT-4) generates an appropriate response based on the user's emotion and input. The input is emotion data and text data, and the output is a response text. Specifically, the generative AI model uses the prompt sentence to generate a response. For example, a response such as "Today was tough. Take a break and relax" may be generated.

[0519] Step 5:

[0520] The server passes the generated text response to speech synthesis software, which (e.g., Amazon Polly) converts the text data into speech data. The input is the response text, and the output is the speech data. Specifically, the speech synthesis algorithm converts the text into a speech waveform.

[0521] Step 6:

[0522] The terminal (robot) plays the generated voice data to the user through a speaker. The input is voice data, and the output is the voice that the user hears. Specifically, the speaker converts the digital voice signal into analog voice and tells the user, "Today was tough. Let's take a break and relax."

[0523] In this way, the system performs specific processing at each step and provides appropriate responses to the user.

[0524] (Application example 1)

[0525] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0526] In modern society, the number of single people and elderly people living alone is increasing, and these people often feel lonely and anxious. In addition, they often have no one to talk to about their troubles and worries in their daily lives, and they lack mental support. Furthermore, it is difficult to propose appropriate products and services that reflect their emotions, which makes it difficult to increase user satisfaction.

[0527] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server is an interactive robot incorporating a generative AI, and is a system for selling to single people and elderly people living alone. The system includes: means for the robot to respond to everything from worries to casual everyday conversations, things to research, and purchasing desired products; means for enabling the robot's personality to be set in advance (age, conversational style); means for learning the user's hobbies, preferences, way of thinking, and outlook on life through repeated conversations; means for analyzing the user's voice input and recognizing emotions; means for generating appropriate responses based on the recognized emotions; and means for suggesting appropriate products and services in accordance with the user's emotions. This makes it possible to provide appropriate responses and suggest products and services in accordance with the user's emotions, thereby providing psychological support and increasing user satisfaction.

[0528] "Generative AI" is an artificial intelligence technology that generates responses in natural language based on user input.

[0529] An "interactive robot" is a robot that uses voice recognition and natural language processing to converse with users.

[0530] "Single person" refers to someone who lives alone.

[0531] "Elderly people living alone" refers to elderly people who live alone.

[0532] "Personality setting" refers to setting the robot's attributes, such as its age and conversational style, in advance.

[0533] "Hobbies and tastes" refers to the user's preferences and interests.

[0534] "Emotion recognition" is a technology that analyzes emotions from a user's tone of voice, speaking style, and choice of words.

[0535] "Response generation" refers to generating appropriate responses based on the user's input and emotions.

[0536] "Product proposal" refers to proposing appropriate products and services based on the user's emotions, hobbies, and preferences.

[0537] "Voice input" refers to the voice that a user makes through a microphone.

[0538] "Internet connection function" refers to the function that allows the robot to connect to the Internet to obtain information and use services.

[0539] The system for implementing this invention uses an interactive robot incorporating generative AI. The system is marketed to single people and elderly people living alone, and can handle everything from worries to everyday conversations, things they want to research, and even purchasing products they want to buy. Specific embodiments of this system are described below.

[0540] 1. System Configuration

[0541] The system consists of the following main components:

[0542] Hardware: Smartphone with microphone

[0543] Software: Python, SpeechRecognition library, OpenAI API, requests library

[0544] 2. Voice Recognition

[0545] The device uses the SpeechRecognition library to convert the user's voice input into text, recognizing the user's speech through the microphone and analyzing its content.

[0546] 3. Emotion recognition

[0547] The server sends the converted voice input to an emotion analysis API to recognize the user's emotions. Emotion recognition is a technology that analyzes emotions from the user's tone of voice, speaking style, and choice of words.

[0548] 4. Response Generation

[0549] The server generates an appropriate response based on the recognized emotion, using a generative AI model to generate a response in natural language based on the user's input and emotion.

[0550] 5. Product proposal

[0551] The server suggests appropriate products and services based on the user's emotions and preferences. For example, if a user says, "I'm tired today, so I want to eat something delicious," the sentiment analysis API recognizes this as "sad." The server suggests "ramen restaurants" and sends a prompt to the generative AI model saying, "Please suggest a menu that you would recommend when the user is in a sad mood."

[0552] Specific examples

[0553] For example, if a user says, "I'm tired today, so I want to eat something delicious," the sentiment analysis API recognizes this as "sad." The server suggests "ramen restaurants" and sends a prompt to the generative AI model saying, "Please suggest a menu item that you would recommend when the user is in a sad mood." The generative AI model might suggest "pork bone ramen."

[0554] Prompt Sentence Examples

[0555] Suggest menu recommendations when the user is feeling sad.

[0556] In this way, the system can provide appropriate responses and product / service suggestions based on the user's emotions, provide psychological support, and increase user satisfaction.

[0557] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0558] Step 1:

[0559] The device receives the user's voice input through a microphone. The user may say, "I'm tired today, so I want to eat something delicious." This voice data becomes the input.

[0560] Step 2:

[0561] The device uses the SpeechRecognition library to convert the voice data into text, analyzes the voice data, and outputs the text "I'm tired today, so I want to eat something delicious."

[0562] Step 3:

[0563] The server sends the converted voice input to the emotion analysis API. Using the text data as input, the emotion analysis API analyzes the user's emotions. The emotion data "sad" is output as the analysis result.

[0564] Step 4:

[0565] The server generates an appropriate response based on the emotion data. Using a generative AI model, the server takes the emotion data "sad" as input and generates a prompt sentence: "Please suggest a menu item that you would recommend when the user is feeling sad." This prompt sentence is the output.

[0566] Step 5:

[0567] The server sends the generated prompt sentence to the generative AI model. Using the prompt sentence as input, the generative AI model generates an appropriate response. For example, it outputs response data such as "I recommend the pork bone ramen."

[0568] Step 6:

[0569] The server sends the generated response data to the terminal. The terminal responds to the user by voice using the response data "We recommend the pork bone ramen." The user receives the voice response "We recommend the pork bone ramen."

[0570] In this way, the system can provide appropriate responses and product / service suggestions based on the user's emotions, provide psychological support, and increase user satisfaction.

[0571] Example 2

[0572] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0573] In modern society, the number of single people and elderly people living alone is increasing, and these people often feel lonely and anxious. Furthermore, if these people are in an emergency, it is difficult for them to receive prompt and appropriate assistance. Furthermore, if the conversations provided by interactive robots are monotonous, users may become bored and stop using the robot.

[0574] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0575] In this invention, the server is a conversational robot incorporating a generative AI, a system for marketing to single people and elderly people living alone. It includes: a means for the robot to respond to everything from worries to casual everyday conversations, things to research, and purchasing desired products; a means for setting the robot's personality (age, conversational style) in advance; a means for learning the user's hobbies, preferences, way of thinking, and outlook on life through repeated conversations; a means for analyzing the user's emotional state; and a means for notifying medical institutions and local governments based on the results of the emotional analysis. This reduces the user's feelings of loneliness and mental anxiety and enables rapid response to emergencies. Furthermore, including a means for engaging the user in conversations encourages continued use of the robot.

[0576] "Generative AI" is a system that automatically generates data and information using artificial intelligence technology.

[0577] An "interactive robot" is a robot that can provide information and receive instructions through conversation with a user.

[0578] "Single person" refers to someone who lives alone.

[0579] "Elderly people living alone" refers to elderly people who live alone.

[0580] "Personality setting" refers to setting a robot's characteristics, such as its age and how it speaks, in advance.

[0581] "Hobbies and tastes" refers to the activities and interests that an individual has.

[0582] "Emotional state" refers to the type and intensity of the emotion the user is feeling.

[0583] "Sentiment analysis" refers to the process of identifying and evaluating the emotions expressed by users through their words and actions.

[0584] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[0585] "Local government" refers to a local public body responsible for local administration.

[0586] "Notification" refers to the act of communicating specific information to another institution or individual.

[0587] A "secure communication protocol" refers to a communication protocol for securely sending and receiving data.

[0588] This invention is a system that provides a variety of services to single people and elderly people living alone using an interactive robot incorporating a generation AI. Specific embodiments of this system will be described below.

[0589] 1. Program Generation

[0590] The server generates a program that contains the robot's personality settings and emotion engine functions. This program is developed using a programming language such as Python or Java (registered trademark).

[0591] 2. Program processing explanation

[0592] The server executes the generated program and performs the following processes.

[0593] Personality setting function

[0594] The user accesses the robot's personality setting screen using a device (such as a smartphone or PC). Here, the user selects the robot's age and conversation style. For example, it is possible to set it to speak in a youthful manner or use honorific language. The server receives the setting information sent by the user and reflects it in the robot's conversation algorithm. At this time, a natural language processing library (e.g., NLTK or spaCy) is used to adjust the conversation style.

[0595] Emotion Engine

[0596] The robot analyzes the user's emotional state through conversation with the user. For emotion analysis, it uses an emotion recognition API (e.g., Microsoft® Azure® Text Analytics API). If the user continues to show sad emotions for a long period of time, the robot sends that information to a server. The server then contacts medical institutions or local governments based on the received information. This communication uses a secure communication protocol (e.g., HTTPS).

[0597] 3. Examples of concrete examples and prompts

[0598] Specific examples

[0599] When a user says to the robot, "I don't feel well today," the robot responds in a youthful voice, "What's wrong? Is something wrong?"

[0600] If the user repeatedly says, "I've been feeling sad lately," the robot analyzes this information using its emotion engine and reports it to the server, which then notifies a medical institution so that appropriate assistance can be provided.

[0601] Prompt Sentence Examples

[0602] "Please set the robot's conversation style to a younger person."

[0603] "Set it to notify medical institutions if the user expresses sad emotions."

[0604] In this way, users can enjoy conversations with robots according to their preferences, while at the same time managing their mental health appropriately.

[0605] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0606] Step 1:

[0607] The user sets the robot's personality.

[0608] The user accesses the robot's personality setting screen using a terminal, where they can select the robot's age and conversation style. For example, they can set it to speak in a youthful manner or use polite language.

[0609] Input: Configuration information entered by the user on the device (age, conversational style).

[0610] Output: The configuration information is sent to the server.

[0611] What happens: The user opens the app on their device, goes to the settings screen, selects "Speak like a young person," and turns on the "Use honorific language" option.

[0612] Step 2:

[0613] The server receives the configuration information and reflects it in the robot's conversation algorithm.

[0614] The server receives the user-submitted configuration information and adjusts the robot's conversation algorithm, changing the conversation style using natural language processing libraries (e.g., NLTK and spaCy).

[0615] Input: Configuration information submitted by the user.

[0616] Output: A tuned conversation algorithm.

[0617] What it does: The server saves the user's preferences in a database and uses a natural language processing library to change the robot's conversational style to sound more youthful.

[0618] Step 3:

[0619] The user starts a conversation with the robot.

[0620] The user starts a conversation with the robot, which reflects the user's settings. The robot then conducts the conversation based on the user's settings.

[0621] Input: What the user says.

[0622] Output: The robot's response.

[0623] Specific behavior: When the user says "Hello" to the robot, the robot responds in a youthful voice with "Yay, how are you?"

[0624] Step 4:

[0625] The robot analyzes the user's emotional state.

[0626] The robot analyzes the user's emotional state through conversation with the user, using an emotion recognition API (e.g., Microsoft Azure's Text Analytics API).

[0627] Input: What the user says.

[0628] Output: Emotion analysis results.

[0629] Specific behavior: When a user says, "I've been feeling sad lately," the robot calls the emotion recognition API and analyzes the user's emotional state as "sad."

[0630] Step 5:

[0631] The server receives the emotion analysis results and notifies medical institutions and local governments as necessary.

[0632] The server receives the emotion analysis results sent from the robot. If the user continues to show sad emotions for a long period of time, the server notifies the information to medical institutions and local governments. A secure communication protocol (e.g., HTTPS) is used.

[0633] Input: Sentiment analysis results.

[0634] Output: Notification to medical institutions and local authorities.

[0635] Specific operation: The server stores the emotion analysis results in a database, and if the "sad" emotion persists for a certain period of time, it notifies the medical institution. The server reports the user's emotional state to the medical institution using HTTPS.

[0636] (Application example 2)

[0637] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0638] In modern society, the number of single people and elderly people living alone is increasing, and these people often feel lonely and anxious. In addition, since they may not be able to respond appropriately in an emergency, there is a need to properly manage their mental health and provide prompt support when necessary. Furthermore, there is a need to provide interactive robots that users can use continuously without getting bored.

[0639] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0640] In this invention, the server is an interactive robot incorporating a generative AI, and is a system for selling to single people and elderly people living alone. It includes means for the robot to respond to everything from worries to casual everyday conversations, things you want to research, and purchasing products you want to buy, means for setting the robot's personality in advance (age, conversation style), means for learning the user's hobbies, tastes, way of thinking, and outlook on life through repeated conversations, means for monitoring the user's emotional state in real time, and means for automatically sending a notification to an emergency contact if the user feels anxiety or fear. This makes it possible to appropriately manage the user's mental health and provide prompt support when necessary.

[0641] "Generative AI" is a system that uses artificial intelligence technology to generate new information and content from data.

[0642] An "interactive robot" is a robot that can have a natural conversation with a user.

[0643] "Single person" refers to someone who lives alone.

[0644] "Elderly people living alone" refers to elderly people who live alone.

[0645] A "system" is a system in which multiple elements work together to perform a specific function.

[0646] "Personality setting" refers to setting the robot's age and conversation style in advance.

[0647] "Hobbies" refers to the things and activities that an individual likes.

[0648] "Emotional state" refers to a user's current feelings or mood.

[0649] "Real time" refers to processing or responding immediately in the current ongoing time.

[0650] "Monitoring" refers to the continuous observation of a particular condition or situation.

[0651] "Emergency Contact" refers to the contact information of a person who should be contacted in an emergency.

[0652] "Notification" refers to sending a message or signal to inform others of specific information.

[0653] This invention is a system that uses an interactive robot incorporating generative AI to appropriately manage the mental health of single people and elderly people living alone and provide prompt support as needed. Specific embodiments of this system are described below.

[0654] System Configuration

[0655] The system consists of the following main components:

[0656] 1. Conversational robots: Robots that can have natural conversations with users and incorporate generative AI.

[0657] 2. Emotion monitoring device: Using a smartphone's camera and microphone, it analyzes the user's facial expressions and tone of voice in real time.

[0658] 3. Notification system: A system that automatically sends notifications to emergency contacts if a user feels anxious or scared.

[0659] Hardware and software used

[0660] Hardware: Smartphone (camera, microphone)

[0661] Software: OpenCV (image processing library), Keras (deep learning library), Twilio (SMS sending service)

[0662] Data processing and calculation

[0663] 1. Emotion monitoring: Capture video in real time using a smartphone camera, detect faces using OpenCV, and input the detected face images into a Keras emotion recognition model to analyze the user's emotional state.

[0664] 2. Anomaly detection: If the user expresses emotions such as "Fear" or "Sad," an SMS will be sent to emergency contacts using Twilio.

[0665] 3. Personality setting: Users can set the personality of the conversational robot within the app, such as a friendly young person, a strict adult, or a polite character who uses honorific language.

[0666] Specific examples

[0667] For example, if a user feels anxious while alone at night, the emotion monitoring device will detect that emotion and the notification system will send a notification to family members or a security company, thus helping to properly manage the user's mental health and providing prompt assistance when needed.

[0668] Prompt Sentence Examples

[0669] Develop an application that automatically sends a notification to an emergency contact if the user feels anxious or scared. Emotion recognition will be performed using the smartphone camera, and emotions will be monitored in real time. If an abnormality is detected, an SMS will be sent using Twilio.

[0670] Thus, specific embodiments for carrying out the invention have been shown.

[0671] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0672] Step 1:

[0673] A user launches a smartphone application. The user's face is captured by the camera as input. The smartphone camera captures video in real time and detects the face using OpenCV. The output is an image of the detected face.

[0674] Step 2:

[0675] The server inputs the detected face image into the Keras emotion recognition model. The face image data is passed to the model as input. The server uses the emotion recognition model to analyze the facial expression and estimate the user's emotional state. The output is the emotional state (e.g., Fear, Sad, Happy, etc.).

[0676] Step 3:

[0677] The server evaluates the estimated emotional state. As input, the emotional state data is evaluated. The server determines whether the emotional state is "Fear" or "Sad." As output, an abnormal emotion flag (True / False) is obtained.

[0678] Step 4:

[0679] If the server determines that the abnormal emotion flag is true, it uses Twilio to send an SMS to the emergency contact. The abnormal emotion flag and the emergency contact information are passed as input. The server calls the Twilio API and sends a message to the emergency contact saying, "The user is experiencing anxiety or fear. Immediate action is required." The status of the SMS transmission is obtained as output.

[0680] Step 5:

[0681] The server logs the status of SMS sending. As input, the status of SMS sending is passed. The server writes the sending result to a log file for later review. As output, the log file is updated.

[0682] Step 6:

[0683] The user sets the personality of the conversational robot. The input is the personality setting selected by the user in the app (e.g., young person, strict adult, polite character using honorific language, etc.). The device applies the selected setting to the robot and changes its conversation style. The output is a conversational robot that reflects the setting.

[0684] Step 7:

[0685] The user starts a conversation with the interactive robot. The user's words are passed to the robot via a microphone as input. The device uses a generative AI to generate an appropriate response based on the user's words and responds to the user. The output is a natural continuation of the conversation.

[0686] In this way, the system monitors the user's emotional state in real time and sends notifications to emergency contacts as needed to properly manage the user's mental health.

[0687] Example 3

[0688] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0689] When using conversational robots for single people or elderly people living alone, it is necessary to properly understand the user's hobbies, tastes, and health conditions and provide personalized conversations. It is also important to adjust the conversation so that the user does not get bored and to respond quickly to emergencies. To solve these issues, a system is needed in which the robot learns through conversations with the user and responds appropriately.

[0690] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0691] In this invention, the server includes means for collecting conversation data with the user, extracting keywords using a generative AI model, and creating a user profile; means for collecting information on the user's health condition and lifestyle and reporting it to medical institutions or local governments as necessary; and means for adjusting the conversation when the user feels bored. This makes it possible to properly understand the user's hobbies, preferences, and health condition and provide personalized conversation. It also makes it possible to adjust the conversation so that the user does not become bored and to respond quickly to emergencies.

[0692] A "generative AI model" is an artificial intelligence technology that analyzes conversation data with users, extracts keywords, and creates user profiles.

[0693] An "interactive robot" is a robot that can converse with users, understand their hobbies, preferences, and health status, and respond appropriately.

[0694] A "profile" is data that compiles information such as a user's hobbies, preferences, way of thinking, and health status.

[0695] "Keywords" are important words and phrases extracted from conversations with users.

[0696] "Health status" refers to information about the user's physical condition, medical history, and daily health.

[0697] "Living conditions" refers to information about the user's daily life, such as lifestyle habits and living environment.

[0698] A "medical institution" is a facility that provides medical services, such as a hospital or clinic.

[0699] A "local government" is a government body responsible for administering a region.

[0700] A "personalized conversation" is a conversation that is individually tailored based on a user's profile.

[0701] An "emergency" is a situation in which a user requires immediate assistance, such as illness or an accident.

[0702] MODE FOR CARRYING OUT THE INVENTION

[0703] This invention is a system that uses an interactive robot incorporating a generative AI model to provide personalized conversations to single people and elderly people living alone, and monitor their health and living conditions. Specific embodiments of this system are described below.

[0704] server

[0705] The server collects conversation data sent from the robot in real time. The collected data is analyzed using a generative AI model to extract important keywords. For example, if a user says, "I went to the movies recently," the conversation data is sent to the server, which then extracts the keyword "movie."

[0706] Next, the server creates a user profile based on the extracted keywords. The profile records the user's hobbies, preferences, way of thinking, health status, etc. For example, if the keyword "movies" appears frequently, the profile will record that the user is a movie lover.

[0707] The server also collects information about the user's health and living conditions and reports it to medical institutions and local governments as necessary. For example, if a user says, "I have a headache today," that information is sent to the server, which then takes steps to contact a medical institution.

[0708] robot

[0709] The robot starts a conversation when the user speaks to it. For example, if the user says "Hello," the robot responds with "Hello, what would you like to talk about today?" The robot provides a conversation tailored to the user based on the profile information provided by the server. For example, if the user's profile indicates that they like movies, the robot will ask, "What movies have you seen recently?"

[0710] The robot also monitors the user's health condition during the conversation. For example, if the user says, "I have a headache today," the robot will suggest, "That's worrying. Should I contact a medical institution?" The robot also has the ability to adjust the conversation if it senses that the user is bored. For example, if the user says, "I'm a bit bored," the robot will change the topic by saying, "Well, how about we tell you something interesting?"

[0711] User

[0712] Users can start a conversation by talking to the robot. For example, they can say, "How is your day?" and the robot will respond. Users can share their hobbies and interests through conversation with the robot. For example, by saying, "I recently went on a trip," the robot will send that information to the server and update their profile.

[0713] If the user feels unwell, they can tell the robot. For example, if they say, "I'm not feeling well today," the robot will suggest an appropriate response. Furthermore, if an emergency occurs, the user can ask the robot for help. For example, if they say, "Help me," the robot will send the information to the server and contact a medical institution or local government.

[0714] Examples and prompts

[0715] Specific examples

[0716] 1. When users talk about travel

[0717] Robot: "Have you traveled anywhere recently?"

[0718] User: "Yes, I went to Kyoto last week."

[0719] Robot: "Kyoto is a wonderful place. What impressed you the most about it?"

[0720] 2. If the user complains of feeling unwell

[0721] Robot: "How are you feeling today?"

[0722] User: "I have a bit of a headache."

[0723] Robot: "That's concerning. May I contact medical help?"

[0724] Prompt Sentence Examples

[0725] 1. Learning about hobbies and tastes

[0726] "Identify hobbies and interests based on what they've talked about recently."

[0727] 2. Reporting on health status

[0728] "If a user complains of feeling unwell, please explain the procedure for reporting this information to a medical institution."

[0729] 3. Moderate the conversation

[0730] "Please explain how you would moderate the conversation if you sense the patron is bored."

[0731] This system enables the robot to provide more personalized services to the user and increase the user's satisfaction. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0732] Step 1:

[0733] The server collects conversation data between the robot and the user in real time.

[0734] Input: Conversation data from the robot

[0735] Output: Collected conversation data

[0736] Specific operation: For example, when a user says "hello," the robot converts the voice data into text data and sends it to the server.

[0737] Step 2:

[0738] The server extracts important keywords from the collected conversation data and uses a generative AI model to understand the context of the conversation and identify the keywords.

[0739] Input: Collected conversation data

[0740] Output: Extracted keywords

[0741] Specific operation: For example, extract the keyword "movie" from the conversation data "I went to see a movie recently."

[0742] Step 3:

[0743] The server creates a user profile based on the extracted keywords.

[0744] Input: Extracted keywords

[0745] Output: User profile

[0746] Specific operation: For example, if the keyword "movie" appears frequently, record in the user's profile that they are a movie lover.

[0747] Step 4:

[0748] The server periodically updates the profile based on new conversation data with the user.

[0749] Input: New conversation data

[0750] Output: Updated profile

[0751] Specific behavior: For example, if a user says, "I recently went on a trip," the information "I love traveling" will be added to their profile.

[0752] Step 5:

[0753] The robot will start a conversation when the user speaks to it.

[0754] Input: Voice input from the user

[0755] Output: Robot response

[0756] Specific behavior: For example, when the user says "Hello," the robot responds, "Hello, what would you like to talk about today?"

[0757] Step 6:

[0758] The robot provides conversation tailored to the user based on the profile information provided by the server.

[0759] Input: Profile information from the server

[0760] Output: Personalized conversation

[0761] Specific behavior: For example, if the user's profile records that they like movies, the robot will ask, "What movies have you seen recently?"

[0762] Step 7:

[0763] The robot monitors the user's health status through conversation.

[0764] Input: User's health-related comments

[0765] Output: Health monitoring results

[0766] Specific behavior: For example, if a user says, "I have a headache today," the robot will suggest, "That's worrying. Should I contact a medical institution?"

[0767] Step 8:

[0768] The robot will adjust the conversation if it senses that the user is bored.

[0769] Input: User's comment about boredom

[0770] Output: Adjusted conversation

[0771] Specific behavior: For example, if the user says, "This is a bit boring," the robot will change the topic by saying, "Well, how about we tell you something interesting?"

[0772] Step 9:

[0773] If the user experiences an emergency, the robot will send the information to a server and contact medical institutions or local authorities.

[0774] Input: User's statement regarding the emergency

[0775] Output: Sending emergency information to the server and contacting medical institutions and local authorities

[0776] Specific behavior: For example, if a user says "help me," the robot immediately sends information to the server and prompts the appropriate response.

[0777] (Application example 3)

[0778] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0779] There is a need to solve problems faced by single people and elderly people living alone, such as loneliness, delayed response in emergencies, and boredom in daily life. There is also a need for a system that can provide conversations tailored to the user's hobbies and ways of thinking, and that users can use for a long time without getting bored. Furthermore, there is a need for a system that can grasp the user's health condition and lifestyle patterns, and respond quickly when abnormalities occur.

[0780] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0781] In this invention, the server is a conversational robot incorporating generative AI, a system for marketing to single people and elderly people living alone. It includes: a means for the robot to respond to everything from worries to casual everyday conversations, things to research, and purchasing desired products; a means for setting the robot's personality (age, conversational style) in advance; a means for learning the user's hobbies, preferences, way of thinking, and outlook on life through repeated conversations; a means for learning the user's lifestyle patterns and health status, and contacting a medical institution or local government if abnormal behavior or poor health is detected; and a means for adjusting the conversation if the user feels bored. This reduces the user's sense of loneliness and enables rapid response in emergencies. Furthermore, by keeping the user interested, the system can be used for long periods of time.

[0782] "Generative AI" is a system that uses artificial intelligence technology to generate data such as text and voice.

[0783] An "interactive robot" is a robot that can have a natural conversation with a user.

[0784] "Single person" refers to someone who lives alone.

[0785] "Elderly people living alone" refers to elderly people who live alone.

[0786] A "system" is a set of mechanisms in which multiple elements function in conjunction with one another.

[0787] "Problems" refer to problems or difficult situations that users face.

[0788] "Daily conversation" refers to general conversation in everyday life.

[0789] "What you want to find out" refers to the information or data that the user wants to know.

[0790] The "item you want to buy" refers to an item that you want to purchase.

[0791] "Personality setting" refers to setting a robot's characteristics, such as its age and how it speaks, in advance.

[0792] "Hobbies and interests" refers to the activities and areas of interest that users prefer.

[0793] "Mindset" refers to the user's thought patterns and values.

[0794] "Life perspective" refers to the user's perspective and feelings about life.

[0795] "Lifestyle patterns" refer to the user's daily habits and behavioral tendencies.

[0796] "Health Status" refers to the state of a user's physical and mental health.

[0797] "Abnormal behavior" refers to actions or reactions that are different from the norm.

[0798] "Illness" refers to physical discomfort or signs of illness felt by the user.

[0799] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[0800] "Local government" refers to a local public body responsible for local administration.

[0801] "To communicate" refers to the use of communication means to convey information.

[0802] "Boredom" refers to a state in which time seems to drag on without interest or stimulation.

[0803] "Adjusting the conversation" means changing the topic and speaking style to suit the user's interests and mood.

[0804] The system for implementing this invention uses an interactive robot incorporating generative AI. The system is marketed to single people and elderly people living alone, and provides various functions through conversations with the user. Specific embodiments of the system are described below.

[0805] System configuration

[0806] The system consists of the following main components:

[0807] 1. Conversational robot: A robot that can have natural conversations with users.

[0808] 2. Generative AI model: An artificial intelligence technology that analyzes user conversations and generates appropriate responses.

[0809] 3. Server: This is the central system that processes and stores data.

[0810] 4. Communication means: A network for data communication between the robot and the server.

[0811] Program processing

[0812] The server analyzes the conversation with the user using the generative AI model and executes the following measures:

[0813] 1. Creating a user profile: Learn about the user's hobbies, preferences, way of thinking, and outlook on life from conversation history and create a profile.

[0814] 2. Anomaly detection and notification: Learns the user's lifestyle patterns and health condition, and contacts medical institutions or local governments if it detects abnormal behavior or poor health.

[0815] 3. Conversation adjustment: If you sense that the user is bored, adjust the conversation to keep them interested.

[0816] Hardware and software used

[0817] Hardware: Smartphones, interactive robots

[0818] Software: Python, OpenAI API, requests library

[0819] Data processing and calculation

[0820] The server collects conversation history with the user and performs the following data processing and calculations using a generative AI model.

[0821] Analysis of conversation history: Extract keywords from users' comments and create a profile.

[0822] Anomaly detection: Analyzes user behavior patterns and notifies you if an anomaly is detected.

[0823] Conversation generation: Generates appropriate conversation content based on the user's profile and current mood.

[0824] Specific examples

[0825] For example, if a user says, "I haven't been feeling well lately," the server will analyze that information and contact a medical institution. If a user says, "I'm bored," the server will provide new topics of conversation, such as, "What interesting movies have you seen recently?"

[0826] Prompt Sentence Examples

[0827] Below is an example of a prompt sentence to input to the generative AI model.

[0828] Create a user profile using the following conversation history:

[0829] "I've been wanting to go on a trip lately."

[0830] "What kind of place do you like?"

[0831] "I like places with a lot of nature."

[0832] User Profile:

[0833] In this way, the system can support users' daily lives and respond quickly in emergencies.

[0834] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0835] Step 1:

[0836] The server collects conversation histories with users.

[0837] Input: User statement

[0838] Output: Conversation history data

[0839] Specific operation: When a user speaks to the interactive robot, their speech is converted into text data using speech recognition technology and sent to the server, which then saves this text data as a conversation history.

[0840] Step 2:

[0841] The server extracts keywords from the conversation history and creates a user profile using a generative AI model.

[0842] Input: Conversation history data

[0843] Output: User profile

[0844] How it works: The server analyzes conversation history data and extracts important keywords. It then inputs prompts into a generative AI model to generate a profile that reflects the user's interests, tastes, and way of thinking.

[0845] Step 3:

[0846] The server learns the user's lifestyle patterns and health status, and detects abnormal behavior or poor health.

[0847] Input: User profile, conversation history data

[0848] Output: Anomaly detection results

[0849] How it works: The server models the user's lifestyle patterns and health status based on their profile and conversation history. If abnormal behavior or poor health is detected, it generates an anomaly detection result.

[0850] Step 4:

[0851] The server contacts medical institutions and local governments based on the abnormality detection results.

[0852] Input: Anomaly detection results

[0853] Output: Information message

[0854] Specific operation: When an anomaly is detected, the server sends a notification message to pre-defined contacts (medical institutions or local governments) containing the user's profile and details of the anomaly.

[0855] Step 5:

[0856] The server moderates the conversation if it senses that the user is bored.

[0857] Input: User profile, current mood

[0858] Output: Adjusted conversation

[0859] How it works: Based on the user's profile and current mood, the server inputs prompts into the generative AI model to generate new, interesting conversation content, which is then provided to the user through the conversational robot.

[0860] Step 6:

[0861] The server provides the generated conversation content to the user to continue the conversation.

[0862] Input: Adjusted conversation

[0863] Output: User statement

[0864] Specific operation: The server sends the generated conversation content to the interactive robot, which then verbally conveys it to the user. When the user makes a new comment, that comment is sent back to the server and added to the conversation history.

[0865] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0866] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0867] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0868] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0869] [Second embodiment]

[0870] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0871] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0872] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0873] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0874] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0875] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0876] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0877] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0878] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0879] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0880] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0881] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0882] "Example 1"

[0883] One aspect of the present invention is a conversational robot incorporating a generative AI. This robot is marketed to single people and elderly people living alone, and can handle everything from worries to everyday conversations, things to look up, and even purchasing products. Specifically, the robot has a voice recognition function and responds by analyzing voice input from the user. It can also purchase products and search for information via its internet connection.

[0884] "Example 2"

[0885] Furthermore, the robot's personality can be set in advance. Specifically, users can set the robot's age and conversation style. For example, they can set the robot to speak like a young person or to use honorific language. This allows users to enjoy conversations with the robot according to their preferences.

[0886] "Example 3"

[0887] Furthermore, through repeated conversations, the robot learns about the user's hobbies, preferences, way of thinking, and outlook on life. Specifically, the robot extracts keywords from conversations with the user and analyzes them to create a profile of the user. For example, if the user often talks about travel, the robot will learn that the user loves traveling. This allows the robot to provide conversations tailored to the user.

[0888] "Example 4"

[0889] Furthermore, the robot will contact medical institutions and local governments based on the content of the conversation to respond to cases of lonely deaths and emergencies. Specifically, the robot will collect information about the user's health and living situation and report it to medical institutions and local governments as necessary. For example, if the user complains of feeling unwell, the robot will contact a medical institution with that information, enabling early response.

[0890] The processing flow of each embodiment will be described below.

[0891] "Example 1"

[0892] Step 1: An interactive robot incorporating generative AI is launched.

[0893] Step 2: The robot uses its voice recognition function to analyze the voice input from the user.

[0894] Step 3: Based on the analysis results, the robot generates an appropriate response and outputs it to the user.

[0895] Step 4: If the user requests to purchase a product or search for information, the robot will perform the corresponding operation via its internet connection function.

[0896] "Example 2"

[0897] Step 1: The user sets the robot's personality from the settings screen.

[0898] Step 2: The user sets the robot's age and conversation style.

[0899] Step 3: Once the setup is complete, the robot will conduct a conversation based on the settings.

[0900] "Example 3"

[0901] Step 1: The robot extracts keywords from the conversation with the user.

[0902] Step 2: Analyze the extracted keywords and create a user profile.

[0903] Step 3: Based on the profile you create, the robot will provide you with a tailored conversation.

[0904] "Example 4"

[0905] Step 1: The robot collects information about the user's health and living conditions.

[0906] Step 2: If the collected information exceeds a certain threshold, the robot reports it to a medical institution or local government.

[0907] Step 3: The medical institution or local government that receives the report will take appropriate action for the user.

[0908] Example 1

[0909] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0910] In modern society, the number of single people and elderly people living alone is increasing, and these people often feel lonely and anxious. In addition, various support is needed in daily life, such as searching for information and purchasing products, but it is difficult for them to do these things alone. Furthermore, due to the lack of means to respond quickly in emergencies, lonely deaths and health problems have become serious issues.

[0911] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0912] In this invention, the server includes a means for converting voice input into text data using a voice recognition function, a means for generating appropriate responses from the text data using a generative AI model, and a means for searching for information and purchasing products using an internet connection function. This allows users to consult with the robot about their worries, engage in everyday conversations, search for information, and purchase products simply by speaking to the robot. The robot can also learn the user's preferences and way of thinking through conversations and provide more personalized responses. Furthermore, in emergencies, the robot can cooperate with medical institutions and local governments to respond quickly.

[0913] "Generative AI" is a system that uses artificial intelligence technology to generate natural language responses from text data.

[0914] An "interactive robot" is a robot that converses with users by combining technologies such as voice recognition, generative AI, and voice synthesis.

[0915] "Single people / elderly people living alone" refers to people who live alone, especially elderly people.

[0916] "Voice recognition function" is a technology that converts voice input into text data.

[0917] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on input text data.

[0918] "Internet connectivity" means the ability to access external information sources and services via the Internet.

[0919] The "voice synthesis function" is a technology that converts text data into voice and outputs it.

[0920] "Personality settings" is a function that allows you to pre-set the robot's characteristics, such as its age and conversational style.

[0921] "Hobbies and tastes" refers to the user's preferences and interests.

[0922] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[0923] "Local government" refers to a government agency responsible for local government administration.

[0924] An "emergency" refers to an urgent situation faced by a user, such as a health problem or accident.

[0925] This invention is a system that provides multifunctional support to single people and elderly people living alone using an interactive robot incorporating generative AI. This system is realized by combining technologies such as voice recognition, generative AI, internet connection, and voice synthesis.

[0926] Hardware and software used

[0927] Speech recognition functionality: Use a speech recognition API (e.g., Google Cloud Speech-to-Text API) to convert voice input into text data.

[0928] Generative AI models: Use generative AI models (e.g., OpenAI's GPT-4) to generate appropriate responses from text data.

[0929] Internet connection functions: Use the Internet connection functions to search for information and purchase products. Specifically, use the Google Search API and Amazon API.

[0930] Speech synthesis function: Uses a speech synthesis API (e.g., Google Cloud Text-to-Speech API) to convert text data into speech and output it.

[0931] Specific operation of the system

[0932] When a user speaks to the robot, it captures the voice through its built-in microphone. It then uses a speech recognition API to convert the voice data into text data. This text data is sent to the server and input into the generative AI model. The generative AI model generates an appropriate response based on the input text data and returns the response to the server.

[0933] The server receives the response from the generative AI model and, if necessary, uses its internet connection to retrieve additional information. For example, if a user asks, "What restaurants are nearby?", the server uses the Google Search API to retrieve information about nearby restaurants. The retrieved information is then combined with the response from the generative AI model to generate the final response.

[0934] The final response is sent from the server to the device, which then uses a speech synthesis API to convert the text data into speech, which is then spoken to the user.

[0935] Specific examples

[0936] Example 1: Everyday conversation

[0937] User: "How's the weather today?"

[0938] Robot: "It's sunny today and the temperature is 25 degrees."

[0939] Example 2: Information search

[0940] User: "What restaurants are nearby?"

[0941] Robot: "There's an Italian restaurant nearby. It has a 4.5 rating."

[0942] Example 3: Purchasing a product

[0943] User: "I want to buy a new smartphone."

[0944] Robot: "The latest smartphone is on sale in the online store. Would you like to buy it?"

[0945] Prompt Sentence Examples

[0946] "When a user speaks to the robot, it uses speech recognition to convert the speech into text, which is then fed into a generative AI model to generate a response. For example, if a user asks, 'What's the weather like today?' the robot will respond, 'It's sunny today, and the temperature is 25 degrees.'"

[0947] In this way, the system of the present invention can meet a variety of user needs by combining voice recognition, generation AI, internet connection, and voice synthesis.

[0948] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0949] Step 1:

[0950] The user provides voice input.

[0951] The user speaks to the robot, for example, asking, "What's the weather like today?" The input is the user's voice, and the output is the audio data captured by the robot's microphone.

[0952] Step 2:

[0953] The device converts the speech to text.

[0954] The device captures the user's voice through the built-in microphone. Then, it uses a speech recognition API to convert the voice data into text data. The input is the voice data, and the output is the text data "What's the weather like today?"

[0955] Step 3:

[0956] The terminal transmits the text data to the server.

[0957] The terminal sends the converted text data to the server. The input is the text data, and the output is the text data sent to the server.

[0958] Step 4:

[0959] The server sends the text data to the generative AI model.

[0960] The server inputs the received text data into the generative AI model. The input is the text data "What's the weather like today?", and the output is the prompt input into the generative AI model.

[0961] Step 5:

[0962] The server receives the response from the generative AI model.

[0963] The generative AI model generates an appropriate response based on the input text data. The server receives this response. The input is the prompt entered into the generative AI model, and the output is the response text: "Today is sunny and the temperature is 25 degrees."

[0964] Step 6:

[0965] The server performs internet searches and purchases as needed.

[0966] When a user requests information search or product purchase, the server uses its Internet connection to obtain the necessary information. For example, it uses the Google Search API to obtain weather information. The input is text data based on the user's request, and the output is the obtained information.

[0967] Step 7:

[0968] The server sends the final response to the terminal.

[0969] The server compiles the responses from the generative AI model and the information obtained from the internet to generate a final response, which is then sent to the terminal. The input is the response text from the generative AI model and the obtained information, and the output is the final response text sent to the terminal.

[0970] Step 8:

[0971] The terminal responds to the user verbally.

[0972] The device converts the final response received from the server into speech using a speech synthesis API. It responds to the user via voice. The input is the final response text, and the output is a voice response to the user. For example, it might respond, "Today is sunny, and the temperature is 25 degrees."

[0973] (Application example 1)

[0974] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0975] Single people and elderly people living alone often feel lonely and anxious in their daily lives, and require support, especially when it comes to ordering meals and managing their health. However, there are only a limited number of effective systems to address these needs. There is also a lack of systems that can provide appropriate suggestions based on the user's hobbies, tastes, and health status. This creates a challenge for users, making it difficult for them to live their lives with peace of mind.

[0976] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0977] In this invention, the server is an interactive robot incorporating a generative AI, a system for selling to single people and elderly people living alone, and includes: means for the robot to respond to everything from worries to casual everyday conversations, things you want to research, and purchasing products you want; means for setting the robot's personality in advance (age, conversational style); means for learning the user's hobbies, preferences, way of thinking, and outlook on life through repeated conversations; means for analyzing voice input and ordering food; means for connecting to a food delivery service via the Internet and completing the order; and means for storing the user's food preferences and allergy information and suggesting appropriate menus. This helps reduce the user's sense of loneliness and anxiety, allowing them to live their lives with peace of mind.

[0978] "Generative AI" refers to algorithms and models that use artificial intelligence techniques to generate new data and information.

[0979] An "interactive robot" is a robot that can converse with humans using voice recognition and natural language processing technology.

[0980] A "single person" is someone who lives alone.

[0981] "Elderly people living alone" refers to elderly people who live alone.

[0982] "Personality setting" refers to setting a robot's characteristics, such as its age and how it speaks, in advance.

[0983] Hobbies and interests are the activities and things that an individual likes.

[0984] "Audio input" refers to obtaining audio as digital data through a device such as a microphone.

[0985] "Ordering food" refers to the act of requesting food from a restaurant or food delivery service.

[0986] "Internet connection" means access to the Internet through a network.

[0987] "Food delivery service" means a service that delivers ordered meals to a specified location.

[0988] "Allergy information" refers to information that indicates an allergic reaction to a specific food ingredient or substance.

[0989] "Menu suggestion" refers to recommending an appropriate meal menu based on the user's preferences and health condition.

[0990] The system for implementing this invention uses an interactive robot incorporating generative AI. The system is marketed to single people and elderly people living alone, and can handle everything from worries to everyday conversations, things you want to research, and even purchasing products you want to buy. Specific embodiments of the system are described below.

[0991] Hardware and Software Configuration

[0992] The system uses the following hardware and software:

[0993] Hardware: Microphone (for voice input), internet-enabled robot

[0994] Software: Python, SpeechRecognition library, OpenAI API, Requests library

[0995] Data processing and calculation

[0996] 1. Acquiring voice input:

[0997] The user provides voice input through a microphone.

[0998] Use the speech recognition library (SpeechRecognition) to convert speech into text data.

[0999] 2. Use of generative AI models:

[1000] The converted text data is sent as a prompt to the generative AI model (OpenAI API).

[1001] The AI ​​model generates an appropriate response based on the prompt.

[1002] 3. Food delivery orders:

[1003] Connect to the API of a food delivery service based on the user's order.

[1004] Use the Requests library to send the order data to the food delivery service and complete the order.

[1005] 4. User information storage and suggestions:

[1006] The database stores the user's dietary preferences and allergy information.

[1007] The next time you order, the system will suggest an appropriate menu item based on the stored information.

[1008] Specific examples

[1009] For example, if a user says, "I want to order a pizza," the system operates as follows:

[1010] 1. Acquiring voice input:

[1011] The user speaks into the microphone, "I'd like to order a pizza."

[1012] The speech recognition library converts this speech into text, generating the text data "I would like to order a pizza."

[1013] 2. Use of generative AI models:

[1014] The generated text data is sent to the OpenAI API as a prompt.

[1015] Example prompt: "The meal the user would like to order is: Pizza."

[1016] The AI ​​model generates the response, "What kind of pizza would you like?"

[1017] 3. Food delivery orders:

[1018] The user responds, "Margherita pizza."

[1019] The system sends this information to the food delivery service's API to complete the order.

[1020] 4. User information storage and suggestions:

[1021] The database stores that the user likes Margherita pizza.

[1022] From the next time onwards, the user will be suggested a Margherita pizza the next time they order pizza.

[1023] In this way, the system can help single people and elderly people living alone to easily order meals, improving the quality of their daily lives.

[1024] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1025] Step 1:

[1026] The user provides voice input through a microphone. When the user says, "I'd like to order a pizza," the microphone captures the voice. The input is the voice data, and the output is the captured voice data.

[1027] Step 2:

[1028] The device uses a speech recognition library (SpeechRecognition) to convert the captured voice data into text data. Specifically, the speech recognition library analyzes the voice data and generates the text data "I would like to order a pizza." The input is voice data, and the output is text data.

[1029] Step 3:

[1030] The device sends the generated text data as a prompt to the generative AI model (OpenAI API). Specifically, the text data "I would like to order pizza" is sent as a prompt to the AI ​​model, which generates an appropriate response. The input is text data, and the output is the response text generated by the AI ​​model.

[1031] Step 4:

[1032] The server receives the response from the generative AI model and responds to the user by voice, asking, "What kind of pizza would you like?" Specifically, the generated response text is converted into voice data using speech synthesis technology and transmitted to the user through a speaker. The input is the response text, and the output is voice data.

[1033] Step 5:

[1034] The user again inputs voice through the microphone and answers "Margherita pizza." The microphone captures the voice and sends it to the device as voice data. The input is voice data, and the output is the captured voice data.

[1035] Step 6:

[1036] The device again uses the speech recognition library to convert the captured voice data into text data. Specifically, the speech recognition library analyzes the voice data and generates the text data "Margherita pizza." The input is voice data, and the output is text data.

[1037] Step 7:

[1038] The server sends the text data "Margherita Pizza" to the food delivery service's API to complete the order. Specifically, it uses the Requests library to send the text data as a POST request to the food delivery service's API endpoint. The input is the text data, and the output is a confirmation of the order completion from the food delivery service.

[1039] Step 8:

[1040] The server stores the user's food preferences and allergy information in a database. Specifically, the user's preference for "Margherita pizza" is saved in the database and used for future orders. The input is the user's preferences and allergy information, and the output is the information saved in the database.

[1041] Step 9:

[1042] From the next time onwards, when the user orders a meal again, the server retrieves the user's preferences and allergy information from the database and suggests an appropriate menu. Specifically, based on the information retrieved from the database, the server suggests to the user, "How about a Margherita pizza?" The input is the information retrieved from the database, and the output is the suggested text.

[1043] Example 2

[1044] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1045] In modern society, the number of single people and elderly people living alone is increasing, and these people often feel lonely. Furthermore, they need various support in their daily lives, such as helping with worries, researching things they want to know, and purchasing products they want to buy, but it is difficult to solve these problems on their own. Furthermore, they are required to learn the user's hobbies, tastes, and ways of thinking, and to provide continuously interesting conversations. Conversational robots are effective in solving these problems, but current systems are not able to adequately address them.

[1046] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1047] In this invention, the server includes means for a user to set the robot's personality using a terminal, means for the terminal to send the setting data to the server, means for the server to send a prompt sentence to the generative AI model, means for the generative AI model to generate a response, means for the server to send the generated response to the robot, and means for the robot to converse with the user, thereby enabling the user to enjoy a conversation with the robot according to their preferences.

[1048] A "generative AI model" is a type of artificial intelligence that generates text data using natural language processing technology.

[1049] A "prompt sentence" is an input sentence that prompts a generative AI model to generate a specific response.

[1050] An "interactive robot" is a robot that can converse with a user through voice or text.

[1051] A "terminal" is a device operated by a user, and includes smartphones, personal computers, etc.

[1052] A "server" is a computer system that processes and stores data over a network.

[1053] "Personality settings" is a feature that allows users to customize the robot's age and conversation style.

[1054] A "response" is text data that a generative AI model generates based on a prompt sentence.

[1055] A "user" is a person who operates an interactive robot and has a conversation with it.

[1056] "Setting data" is information about the robot's personality settings that the user inputs through the terminal.

[1057] A "conversation" is a communication exchange between a user and an interactive robot.

[1058] This invention is a system that uses an interactive robot incorporating a generative AI model to help single people and elderly people living alone reduce their feelings of loneliness and receive support in their daily lives. Specific embodiments of this system are described below.

[1059] Hardware and software used

[1060] Hardware: Robot body, server, user terminal (PC, smartphone, etc.)

[1061] Software: Generative AI models (e.g., models using natural language processing techniques), robot control software, user interface (UI) software

[1062] Data processing and calculation

[1063] 1. The user sets the robot's personality using the device

[1064] Users use the device's UI to set the robot's age and how it will converse.

[1065] Example: A user opens an app on their smartphone and selects "Age: Young" and "Conversational style: Casual" on the settings screen.

[1066] 2. The device sends the configuration data to the server.

[1067] The terminal transmits the setting data entered by the user to the server.

[1068] Example: The device packages configuration data in JSON format and sends it to the server using the HTTPS protocol.

[1069] 3. The server sends a prompt to the generative AI model

[1070] The server sends a prompt to the generative AI model based on the received configuration data.

[1071] Example: The server generates a prompt saying, "Generate a response from a robot that can have casual conversations like a young person," and sends it to the API endpoint of the generative AI model.

[1072] 4. The generative AI model generates a response

[1073] The generative AI model generates the robot's response based on the prompt sentence.

[1074] Example: A generative AI model parses a prompt and generates the response "Hey there! What are you doing today?"

[1075] 5. The server generates a response and sends it to the robot

[1076] The server sends the response received from the generative AI model to the robot.

[1077] Example: The server sends the response data to the robot control software, which outputs it as sound from the robot's speaker.

[1078] 6. The robot will converse with the user

[1079] The robot will then converse with the user based on the received response.

[1080] Example: A robot speaks to the user, "Hey there! What are you doing today?" When the user responds, the robot sends the data back to the server and the process repeats to generate the next response.

[1081] Prompt Sentence Examples

[1082] "Generate responses from a robot that can have casual conversations like a young person."

[1083] "Generate a polite robotic response using honorific language."

[1084] This system allows users to enjoy conversations with robots according to their preferences, reducing feelings of loneliness and providing support for daily life.

[1085] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1086] Step 1:

[1087] The user uses the terminal to set the robot's personality.

[1088] Input: Configuration data that users enter through the device's UI (e.g., age, speech patterns).

[1089] Specific actions: The user opens the app on their smartphone and selects "Age: Young" and "Conversation style: Casual" on the settings screen.

[1090] Output: The configuration data is saved to the device.

[1091] Step 2:

[1092] The terminal sends the setting data to the server.

[1093] Input: Configuration data entered by the user.

[1094] What happens: The device packages the configuration data in JSON format and sends it to the server using the HTTPS protocol.

[1095] Output: The configuration data is sent to the server.

[1096] Step 3:

[1097] The server sends a prompt to the generative AI model.

[1098] Input: The configuration data received by the server.

[1099] Specific operation: The server generates a prompt sentence, "Please generate a response from a robot that can have casual conversation like a young person," and sends it to the API endpoint of the generative AI model.

[1100] Output: The prompt is sent to the generative AI model.

[1101] Step 4:

[1102] A generative AI model generates a response.

[1103] Input: The prompt received by the generative AI model.

[1104] What it does: The generative AI model analyzes the prompt and generates the response, "Hey there! What are you doing today?"

[1105] Output: The generated response is sent back to the server.

[1106] Step 5:

[1107] The server generates a response and sends it to the robot.

[1108] Input: The response returned by the generative AI model.

[1109] Specific operation: The server sends the response data to the robot control software, which outputs it as sound from the robot's speaker.

[1110] Output: The response data is sent to the robot.

[1111] Step 6:

[1112] The robot converses with the user.

[1113] Input: The response data received by the robot.

[1114] What it does: The robot asks the user, "Hey! What are you doing today?" When the user responds, the robot sends the data back to the server and the process repeats, generating the next response.

[1115] Output: The conversation with the user continues.

[1116] (Application example 2)

[1117] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1118] In modern society, the number of single people and elderly people living alone is increasing, and supporting their loneliness and daily life is becoming an important issue. Another problem is the lack of people to talk to to alleviate the worries and anxieties they experience on a daily basis. Furthermore, there is a need to realize more friendly and timeless communication by providing conversations tailored to the user's preferences.

[1119] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server is an interactive robot incorporating a generative AI, and is a system for selling to single people and elderly people living alone. The system includes: means for the robot to respond to everything from worries to casual everyday conversations, things to research, and purchasing products; means for enabling the robot's personality to be set in advance (age, conversational style); means for learning the user's hobbies, tastes, way of thinking, and outlook on life through repeated conversations; means for the user to set the robot's personality using a smart device; means for generating natural conversations using a generative AI model based on the set personality; and means for the robot to speak the generated conversation content to the user. This allows for dialogue tailored to the user's preferences, reducing feelings of loneliness and supporting daily life.

[1120] "Generative AI" is a system that uses artificial intelligence technology to process natural language and generate dialogue with users.

[1121] An "interactive robot" is a robot that can communicate with users through voice and text.

[1122] "Single person" refers to someone who lives alone.

[1123] "Elderly people living alone" refers to elderly people who live alone.

[1124] A "system" is a structure in which multiple elements are combined to achieve a specific function.

[1125] "Worries" refer to problems and anxieties that users experience in their daily lives.

[1126] "Daily conversation" refers to general conversation in everyday life.

[1127] "What you want to know" refers to the information or questions that users want to know.

[1128] "Products you want to buy" refers to items that users are considering purchasing.

[1129] "Personality settings" refers to the user pre-setting the robot's age and conversation style.

[1130] "Hobbies and tastes" refers to the user's interests and preferences.

[1131] "Thoughts" refers to the user's thoughts and opinions.

[1132] "Life philosophy" refers to the user's perspective and values ​​on life.

[1133] "Smart devices" refer to electronic devices that can connect to the Internet, such as smartphones and tablets.

[1134] "Generative AI model" refers to the algorithms and data models used to realize generative AI.

[1135] "Natural conversation" refers to conversations that are generated in a manner that is close to dialogue between humans.

[1136] The system for implementing this invention uses an interactive robot incorporating generative AI. The system is marketed to single people and elderly people living alone, and can handle everything from worries to everyday conversations, research needs, and product purchases. A specific embodiment of this system is described below.

[1137] System configuration

[1138] The system consists of the following main components:

[1139] 1. Conversational robot: A robot that communicates with users through voice and text.

[1140] 2. Smart Device: A device such as a smartphone or tablet that allows the user to set the robot's personality.

[1141] 3. Generative AI models: Algorithms and data models that perform natural language processing and generate user interactions.

[1142] Program processing

[1143] The server implements the system using the following means:

[1144] 1. Personality setting method: The user uses a smart device to set the robot's age and conversation style. For example, if the user sets the robot to "speak casually like a young person," the prompt text will be as follows:

[1145] Speak casually, like you're a young person.

[1146] 2. Conversation generation method: A generative AI model is used based on the set personality to generate natural conversations. The generative AI model used is OpenAI's GPT-3.

[1147] 3. Conversation output means: The robot speaks the generated conversation content to the user, allowing the user to enjoy interacting with the robot.

[1148] Specific examples

[1149] For example, when a user sets a robot's personality using a smartphone, the procedure is as follows:

[1150] 1. The user opens the smartphone application and sets the robot's age and conversation style.

[1151] 2. The configuration information is sent to the server and a prompt is generated.

[1152] 3. Based on the prompt, the generative AI model generates natural conversation.

[1153] 4. The generated conversation content is sent to the robot, which then speaks to the user.

[1154] In this way, the system can provide interactions tailored to the user's preferences, reducing feelings of loneliness and supporting daily life.

[1155] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1156] Step 1:

[1157] The user configures the robot's personality using a smart device. Specifically, the user opens an application on their smartphone or tablet and configures the robot's age and conversation style. The input includes the user's selection of age and conversation style. The output is the configuration information.

[1158] Step 2:

[1159] The terminal sends the user's setting information to the server. Specifically, the setting information is sent from the smart device to the server. The input includes the user's age and conversation style. The server receives the setting information as output.

[1160] Step 3:

[1161] The server generates a prompt sentence based on the configuration information it receives. Specifically, it analyzes the configuration information and creates a prompt sentence suitable for the generative AI model. The input includes the user's configuration information. The output is the generated prompt sentence. For example, a generated prompt sentence might be, "Please speak casually, like a young person."

[1162] Step 4:

[1163] The server uses a generative AI model to generate natural conversations. Specifically, the generated prompt sentence is input into a generative AI model (e.g., OpenAI's GPT-3) to generate the conversation content. The input includes the prompt sentence. The output is the generated conversation content.

[1164] Step 5:

[1165] The server sends the generated conversation content to the robot. Specifically, the server sends the generated conversation content to the interactive robot, and the robot prepares to speak the content to the user. The generated conversation content is included as input. The robot receives the conversation content as output.

[1166] Step 6:

[1167] The robot speaks the generated conversation content to the user. Specifically, the robot uses speech synthesis technology to convey the generated conversation content to the user by voice. The generated conversation content is included as input. The output allows the user to enjoy a dialogue with the robot.

[1168] Example 3

[1169] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1170] There is a need to reduce loneliness among single people and elderly people living alone and to provide support in their daily lives. It is also important to monitor the user's health and living situation and respond quickly to emergencies. Conventional interactive robots have had difficulty fully understanding the user's hobbies and health and responding appropriately. Another challenge is maintaining a conversation so that the user does not get bored.

[1171] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1172] In this invention, the server includes a means for collecting conversations with the user as voice input, a means for converting the collected voice data into text data, and a means for extracting keywords from the converted text data, thereby enabling the server to understand the user's hobbies and health condition and provide appropriate support.

[1173] "Generative AI" is a system that uses artificial intelligence technology to generate data such as text and voice.

[1174] An "interactive robot" is a robot that can converse with users through voice or text.

[1175] "Single people / elderly people living alone" refers to people who live alone, and specifically includes elderly people.

[1176] "Personality setting" refers to setting a robot's characteristics, such as its age and how it speaks, in advance.

[1177] "Hobbies and interests" refers to the activities and areas of interest that users prefer.

[1178] A "profile" is data that compiles information such as a user's hobbies, thoughts, and health status.

[1179] "Voice input" refers to collecting a user's speech as voice data.

[1180] "Text data" refers to data obtained by converting voice data into character information.

[1181] "Keywords" are important words and phrases extracted from conversations.

[1182] "Generative AI model" refers to an algorithm or system that uses artificial intelligence technology to generate text or speech.

[1183] "Audio output" refers to the presentation of generated text or information to the user as audio.

[1184] "Health status" refers to information about the user's physical condition and medical care.

[1185] "Living situation" refers to information about the user's daily life.

[1186] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[1187] "Local government" refers to a local administrative body that plays a role in supporting the lives of residents.

[1188] This invention is a system that uses an interactive robot incorporating a generative AI to help single people and elderly people living alone reduce feelings of loneliness and provide support for their daily lives. Specific embodiments of this system are described below.

[1189] Hardware and software used

[1190] Hardware: Interactive robots (e.g., general robotic platforms)

[1191] Software: Natural language processing engines (e.g., general natural language processing APIs), generative AI models (e.g., general generative AI models)

[1192] Explanation of program processing

[1193] Conversation data collection

[1194] The terminal (interactive robot) collects conversations with the user as voice input. Specifically, when the user speaks to the robot, the robot captures the voice with a microphone and saves it as voice data.

[1195] Keyword extraction

[1196] The device uses speech recognition software to convert the collected voice data into text data. The server then uses a natural language processing engine to extract keywords from the converted text data. For example, if the voice data is converted into text such as "I want to go on a trip recently," keywords such as "travel" and "want to go" are extracted.

[1197] Creating a profile

[1198] The server analyzes the extracted keywords and creates a user profile. For example, if the keyword "travel" appears frequently, the server records the user as "travel lover" in the user's profile.

[1199] Conversational Offerings

[1200] The server uses the generative AI model to generate a conversation based on the user's profile. The device then provides the generated conversation to the user as voice output. For example, the server generates a question such as, "Is there anywhere you'd like to travel to recently?", and the device then verbally replies the question to the user.

[1201] Health monitoring

[1202] The device collects information about the user's health and lifestyle during the conversation. The server analyzes the collected information and checks for any abnormalities. For example, if a user says, "I haven't been feeling well lately," the device sends that information to the server, which then analyzes it.

[1203] Emergency response

[1204] The server will report the results of the health status analysis to medical institutions or local governments as necessary. For example, if the server detects that the user is in poor health, it will automatically contact a medical institution by email or phone.

[1205] Prompt Sentence Examples

[1206] Prompt: "What questions should be generated when a user talks about a recent trip?"

[1207] Example output from a generative AI model: "Is there anywhere you'd like to travel to these days?"

[1208] This system enables the robot to understand the user's hobbies and health condition and take appropriate measures. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1209] Step 1:

[1210] The device collects conversations with users as voice input. Specifically, when a user speaks to the robot, the robot captures the voice with a microphone and saves it as voice data.

[1211] Input: User speech

[1212] Output: Audio data

[1213] Step 2:

[1214] The device uses voice recognition software to convert the collected voice data into text data. For example, the voice recognition software analyzes the voice data and converts it into text such as "I want to go on a trip these days."

[1215] Input: Audio data

[1216] Output: Text data

[1217] Step 3:

[1218] The server then uses a natural language processing engine to extract keywords from the converted text data. Specifically, the natural language processing engine analyzes the text data and extracts keywords such as "travel" and "want to go."

[1219] Input: Text data

[1220] Output: Keywords

[1221] Step 4:

[1222] The server analyzes the extracted keywords and creates a user profile. For example, if the keyword "travel" appears frequently, the server records the user as "travel lover" in the user's profile.

[1223] Input: keyword

[1224] Output: User profile

[1225] Step 5:

[1226] The server uses a generative AI model to generate a conversation based on the user's profile. Specifically, the generative AI model generates a question based on the profile information, such as "Is there anywhere you'd like to travel to recently?"

[1227] Input: User Profile

[1228] Output: Generated conversation text

[1229] Step 6:

[1230] The device provides the generated conversation to the user as a voice output. Specifically, the device converts the generated text into voice using speech synthesis software and conveys the voice to the user.

[1231] Input: Generated conversation text

[1232] Output: Audio output

[1233] Step 7:

[1234] The device collects information about the user's health and lifestyle during conversations. For example, if the user says, "I haven't been feeling well lately," the device collects that information as voice data.

[1235] Input: User speech

[1236] Output: Audio data about health status

[1237] Step 8:

[1238] The server converts the collected voice data about health conditions into text data for analysis. Specifically, voice recognition software converts the voice data into text, and a natural language processing engine analyzes the text.

[1239] Input: Audio data about health conditions

[1240] Output: Text data about health conditions

[1241] Step 9:

[1242] The server will report the results of the health status analysis to medical institutions or local governments as necessary. Specifically, if the server detects an abnormality, it will automatically contact the medical institution by email or phone.

[1243] Input: Text data about health conditions

[1244] Output: Report to medical institutions and local authorities

[1245] (Application example 3)

[1246] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1247] In modern society, the number of single people and elderly people living alone is increasing, and their feelings of loneliness and how to respond in emergencies are becoming major issues. There is also a need to understand their hobbies and health conditions and provide appropriate support. However, conventional interactive robots have had difficulty fully learning about users' hobbies and health conditions and responding appropriately in emergencies.

[1248] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server is an interactive robot incorporating a generative AI, and is a system for selling to single people and elderly people living alone. The system includes: means for the robot to respond to everything from worries to casual everyday conversations, things to research, and purchasing desired products; means for enabling the robot's personality to be set in advance (age, conversational style); means for learning the user's hobbies, tastes, way of thinking, and outlook on life through repeated conversations; means for collecting information on the user's health condition and living situation and reporting it to medical institutions or local governments as necessary; and means for learning the user's hobbies and health condition through daily conversations and contacting medical institutions or local governments in case of an emergency. This makes it possible to understand the user's hobbies and health condition and provide appropriate support.

[1249] "Generative AI" is a system that uses artificial intelligence technology to generate new information and content from data.

[1250] An "interactive robot" is a robot that can have a natural conversation with a user.

[1251] "Single person" refers to someone who lives alone.

[1252] "Elderly people living alone" refers to elderly people who live alone.

[1253] A "system" is a mechanism in which multiple elements are combined to achieve a specific function.

[1254] "Problems" refer to problems or difficult situations that users face.

[1255] "Daily conversation" refers to general conversation in everyday life.

[1256] "What you want to find out" refers to the information or data that the user wants to know.

[1257] The "item you want to buy" refers to an item that you want to purchase.

[1258] "Personality setting" refers to setting a robot's characteristics, such as its age and how it speaks, in advance.

[1259] "Hobbies and tastes" refers to the activities and interests that users prefer.

[1260] "Mindset" refers to the user's thinking tendencies and values.

[1261] "Life perspective" refers to the user's perspective and feelings about life.

[1262] "Health Status" refers to the state of a user's physical and mental health.

[1263] "Living conditions" refers to the environment and circumstances of the user's daily life.

[1264] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[1265] "Local government" refers to a local public body responsible for local administration.

[1266] "Emergency" means a situation in which the User requires immediate attention.

[1267] "Reporting" refers to conveying collected information to relevant authorities.

[1268] The system for implementing this invention uses a conversational robot incorporating generative AI. The system is marketed to single people and elderly people living alone, and can handle everything from worries to everyday conversations, things they want to research, and even purchasing products they want to buy. It also allows the robot's personality (age, conversational style) to be set in advance, and through repeated conversations, it learns the user's hobbies, preferences, way of thinking, and outlook on life.

[1269] Program processing explanation

[1270] Hardware

[1271] Smartphone: A device that users use on a daily basis.

[1272] Server: Processes data and runs generative AI models.

[1273] software

[1274] OpenAI API: Uses generative AI models to analyze conversation data and create user profiles.

[1275] requests library: Sends HTTP requests to contact medical institutions and local authorities in emergencies.

[1276] Data processing and calculation

[1277] 1. Collection of conversation data: Collect users' everyday conversations via smartphones.

[1278] 2. Profile creation: Using the OpenAI API, the user's hobbies and health status are extracted from the collected conversation data to create a profile.

[1279] 3. Emergency response: If a user complains of feeling unwell, the requests library will be used to automatically contact medical institutions and local authorities.

[1280] Specific examples

[1281] For example, if a user says, "I'd like to travel recently, but I haven't been feeling well," the system collects this conversation and uses the OpenAI API to input the following prompt sentence into the generative AI model:

[1282] Prompt Sentence Examples

[1283] Extract the user's hobbies and health status from the following conversation:

[1284] I've been wanting to go on a trip lately, but I haven't been feeling well.

[1285] Based on this prompt, the generative AI model learns that the user is a travel enthusiast and is currently unwell. It then uses the requests library to send a message to a medical institution saying, "The user is feeling unwell. Please take immediate action."

[1286] In this way, the system can understand the user's hobbies and health status and provide appropriate support.

[1287] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1288] Step 1:

[1289] A user has everyday conversations through a smartphone. The input is the user's voice data, which the smartphone converts into text data. The output is text data.

[1290] Step 2:

[1291] The smartphone sends text data to the server. The input is the text data, and the output is the text data sent to the server.

[1292] Step 3:

[1293] The server uses the OpenAI API to extract the user's hobbies and health status from the text data. The input is the text data and a prompt, which is "Please extract the user's hobbies and health status from the following conversation: [text data]." The output is the user's profile information.

[1294] Step 4:

[1295] The server stores the user's profile information in a database. The input is the user's profile information, and the output is the profile information stored in the database.

[1296] Step 5:

[1297] The server analyzes the user's profile information and determines whether emergency response is required. The input is the profile information, and the output is a flag indicating whether emergency response is required.

[1298] Step 6:

[1299] If an emergency response is deemed necessary, the server contacts medical institutions and local authorities using the requests library. The input is the emergency response flag and contact information, and the output is the HTTP request sent.

[1300] Step 7:

[1301] The server records the contact result in a database and notifies the user. The input is the result of the HTTP request, and the output is the result recorded in the database and a notification to the user.

[1302] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1303] "Example 1"

[1304] One embodiment of the present invention is a system that combines an emotion engine with a conversational robot incorporating generative AI. This system recognizes emotions from the user's tone of voice, speaking style, and choice of words, and adjusts the robot's response accordingly. For example, if the user is sad, the robot will choose comforting words, and if the user is happy, it will choose words that share that joy.

[1305] "Example 2"

[1306] The emotion engine also connects with medical institutions and local governments based on the user's emotional state. For example, if the user continues to show sadness for a long period of time, the robot will report that information to medical institutions and local governments. This makes it possible to properly manage the user's mental health and provide appropriate support as needed.

[1307] "Example 3"

[1308] Furthermore, the emotion engine has the ability to adjust the conversation so that the user does not get bored. For example, if the robot senses that the user is bored, it will change the topic or tell a joke to keep the user interested. This prevents conversations with the robot from becoming monotonous and encourages users to continue using the robot for a long period of time.

[1309] The processing flow of each embodiment will be described below.

[1310] "Example 1"

[1311] Step 1: The user talks to the robot.

[1312] Step 2: The emotion engine recognizes the user's emotions from their tone of voice, speaking style, and choice of words.

[1313] Step 3: Adjust the robot's response depending on the emotion recognized by the emotion engine.

[1314] "Example 2"

[1315] Step 1: The emotion engine monitors the user's emotional state over time.

[1316] Step 2: If the user continues to show sad emotions for a long period of time, the emotion engine will report this information to medical institutions and local authorities.

[1317] Step 3: Medical institutions and local authorities receive the report and provide appropriate assistance as needed.

[1318] "Example 3"

[1319] Step 1: The emotion engine monitors the user's emotional state.

[1320] Step 2: If the user feels bored, the emotion engine instructs the robot to change the topic or tell a joke.

[1321] Step 3: The robot follows instructions from the emotion engine and engages in a conversation that engages the user.

[1322] Example 1

[1323] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1324] In modern society, the number of single people and elderly people living alone is increasing, and the loneliness and inconveniences they face in their daily lives are becoming serious problems. In particular, they face challenges such as a lack of everyday conversation skills, not having anyone to talk to about their worries, and difficulty finding necessary information or purchasing products. There is also a need to understand the emotions of these people and respond appropriately, but existing technologies are not sufficient. Furthermore, rapid response in emergencies and cooperation with medical institutions and local governments are also important challenges.

[1325] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1326] In this invention, the server includes a means for analyzing voice input from a user using a voice recognition function, a means for recognizing emotions from the user's tone of voice, speaking style, and vocabulary using an emotion engine, and a means for generating a response based on the user's emotions and input using a generative AI model. This reduces the user's sense of loneliness and eliminates inconveniences in daily life. Providing appropriate responses according to the user's emotions enables more intimate conversations. Furthermore, by including a means for searching for information and purchasing products via an internet connection function, user convenience can be improved. Additionally, by including a means for coordinating with medical institutions and local governments, it becomes possible to respond quickly to emergencies.

[1327] "Generative AI" is a system that uses artificial intelligence technology to generate data such as text and voice.

[1328] An "interactive robot" is a robot that has the ability to converse with humans through voice and text.

[1329] "Single person" refers to someone who lives alone.

[1330] "Elderly people living alone" refers to elderly people who live alone.

[1331] "Voice recognition function" is a technology that converts voice data into text data.

[1332] An "emotion engine" is a technology that analyzes a user's emotions from voice and text.

[1333] A "generative AI model" is an algorithm that uses artificial intelligence techniques to generate responses based on input data.

[1334] "Speech synthesis function" is a technology that converts text data into voice data.

[1335] "Internet connection function" refers to the function of accessing external information and services via the Internet.

[1336] "Information search" is the act of finding necessary information using the Internet or databases.

[1337] "Purchasing a product" refers to the act of ordering and purchasing a product via the Internet.

[1338] "Medical institutions" are facilities that provide medical services, such as hospitals and clinics.

[1339] A "local government" is a local public body responsible for administering a region.

[1340] "Emergency" refers to an urgent or critical situation faced by a User.

[1341] This invention provides a conversational robot incorporating generative AI, and is a system that targets single people and elderly people living alone, providing advice on worries, daily conversation, information searches, product purchases, etc. Furthermore, by combining it with an emotion engine, it can respond according to the user's emotions.

[1342] Hardware and software used

[1343] The terminal (robot) uses the following hardware and software:

[1344] Microphone: A device for capturing audio input

[1345] Speaker: A device for audio output.

[1346] Internet connection module: A device for connecting to an external network

[1347] The server uses the following software:

[1348] Speech recognition software (e.g., Google Cloud Speech-to-Text): converts voice data into text data

[1349] Emotion engine (e.g. IBM Watson Tone Analyzer): Analyzes emotions from voice and text

[1350] Generative AI models (e.g., OpenAI GPT-4): generate responses based on input data

[1351] Speech synthesis software (e.g., Amazon Polly): converts text data into speech data

[1352] Program processing

[1353] The user speaks to the robot, for example, saying, "I'm very tired today."

[1354] The terminal (robot) captures the user's voice using a built-in microphone and transmits the voice data to the server in real time.

[1355] The server passes the received voice data to speech recognition software, which converts the voice data into text data, such as "I'm very tired today."

[1356] The server passes the converted text and voice data to an emotion engine, which analyzes the user's emotions based on their tone of voice, speaking style, and choice of words. For example, it can recognize that the user is tired.

[1357] The server passes the emotion recognition results and text data to a generative AI model, which generates an appropriate response based on the user's emotions and input. For example, the response might be, "You've had a tough day. Take a break and relax."

[1358] The server passes the generated text response to speech synthesis software, which converts the text data into speech data.

[1359] The device (robot) plays the generated voice data back to the user through a speaker, for example, saying, "Today was tough. Let's take a break and relax."

[1360] Examples of concrete examples and prompts

[1361] For example, if a user says, "I'm so tired today," the robot will use its emotion engine to recognize that the user is tired, and the generative AI model will generate a response such as, "You've had a hard day today. Take a break and relax."

[1362] Example prompt sentence:

[1363] "Generate comforting words when the user is tired. The user's input is 'I'm very tired today.'"

[1364] In this way, the system combines voice recognition, emotion recognition, generative AI models, speech synthesis, and internet connectivity to provide appropriate responses to users.

[1365] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1366] Step 1:

[1367] The user speaks to the robot. For example, they say, "I'm very tired today." The terminal (robot) captures the user's voice using a built-in microphone. The input is the user's voice data, and the output is the captured voice data. Specifically, the microphone converts the voice into a digital signal and sends that data to the server.

[1368] Step 2:

[1369] The server passes the received voice data to speech recognition software. The speech recognition software (e.g., Google Cloud Speech-to-Text) converts the voice data into text data. The input is voice data, and the output is text data. Specifically, the speech recognition algorithm analyzes the voice waveform and generates corresponding text. For example, the generated text might say, "I'm very tired today."

[1370] Step 3:

[1371] The server passes the converted text and voice data to an emotion engine. The emotion engine (e.g., IBM Watson Tone Analyzer) analyzes emotions from the user's tone of voice, speaking style, and choice of words. The input is text and voice data, and the output is emotion data. Specifically, the emotion engine analyzes the characteristics of the voice and text and recognizes that the user is tired.

[1372] Step 4:

[1373] The server passes the emotion recognition results and text data to a generative AI model. The generative AI model (e.g., OpenAI GPT-4) generates an appropriate response based on the user's emotion and input. The input is emotion data and text data, and the output is a response text. Specifically, the generative AI model uses the prompt sentence to generate a response. For example, a response such as "Today was tough. Take a break and relax" may be generated.

[1374] Step 5:

[1375] The server passes the generated text response to speech synthesis software, which (e.g., Amazon Polly) converts the text data into speech data. The input is the response text, and the output is the speech data. Specifically, the speech synthesis algorithm converts the text into a speech waveform.

[1376] Step 6:

[1377] The terminal (robot) plays the generated voice data to the user through a speaker. The input is voice data, and the output is the voice that the user hears. Specifically, the speaker converts the digital voice signal into analog voice and tells the user, "Today was tough. Let's take a break and relax."

[1378] In this way, the system performs specific processing at each step and provides appropriate responses to the user.

[1379] (Application example 1)

[1380] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1381] In modern society, the number of single people and elderly people living alone is increasing, and these people often feel lonely and anxious. In addition, they often have no one to talk to about their troubles and worries in their daily lives, and they lack mental support. Furthermore, it is difficult to propose appropriate products and services that reflect their emotions, which makes it difficult to increase user satisfaction.

[1382] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server is an interactive robot incorporating a generative AI, and is a system for selling to single people and elderly people living alone. The system includes: means for the robot to respond to everything from worries to casual everyday conversations, things to research, and purchasing desired products; means for enabling the robot's personality to be set in advance (age, conversational style); means for learning the user's hobbies, preferences, way of thinking, and outlook on life through repeated conversations; means for analyzing the user's voice input and recognizing emotions; means for generating appropriate responses based on the recognized emotions; and means for suggesting appropriate products and services in accordance with the user's emotions. This makes it possible to provide appropriate responses and suggest products and services in accordance with the user's emotions, thereby providing psychological support and increasing user satisfaction.

[1383] "Generative AI" is an artificial intelligence technology that generates responses in natural language based on user input.

[1384] An "interactive robot" is a robot that uses voice recognition and natural language processing to converse with users.

[1385] "Single person" refers to someone who lives alone.

[1386] "Elderly people living alone" refers to elderly people who live alone.

[1387] "Personality setting" refers to setting the robot's attributes, such as its age and conversational style, in advance.

[1388] "Hobbies and tastes" refers to the user's preferences and interests.

[1389] "Emotion recognition" is a technology that analyzes emotions from a user's tone of voice, speaking style, and choice of words.

[1390] "Response generation" refers to generating appropriate responses based on the user's input and emotions.

[1391] "Product proposal" refers to proposing appropriate products and services based on the user's emotions, hobbies, and preferences.

[1392] "Voice input" refers to the voice that a user makes through a microphone.

[1393] "Internet connection function" refers to the function that allows the robot to connect to the Internet to obtain information and use services.

[1394] The system for implementing this invention uses an interactive robot incorporating generative AI. The system is marketed to single people and elderly people living alone, and can handle everything from worries to everyday conversations, things they want to research, and even purchasing products they want to buy. Specific embodiments of this system are described below.

[1395] 1. System Configuration

[1396] The system consists of the following main components:

[1397] Hardware: Smartphone with microphone

[1398] Software: Python, SpeechRecognition library, OpenAI API, requests library

[1399] 2. Voice Recognition

[1400] The device uses the SpeechRecognition library to convert the user's voice input into text, recognizing the user's speech through the microphone and analyzing its content.

[1401] 3. Emotion recognition

[1402] The server sends the converted voice input to an emotion analysis API to recognize the user's emotions. Emotion recognition is a technology that analyzes emotions from the user's tone of voice, speaking style, and choice of words.

[1403] 4. Response Generation

[1404] The server generates an appropriate response based on the recognized emotion, using a generative AI model to generate a response in natural language based on the user's input and emotion.

[1405] 5. Product proposal

[1406] The server suggests appropriate products and services based on the user's emotions and preferences. For example, if a user says, "I'm tired today, so I want to eat something delicious," the sentiment analysis API recognizes this as "sad." The server suggests "ramen restaurants" and sends a prompt to the generative AI model saying, "Please suggest a menu that you would recommend when the user is in a sad mood."

[1407] Specific examples

[1408] For example, if a user says, "I'm tired today, so I want to eat something delicious," the sentiment analysis API recognizes this as "sad." The server suggests "ramen restaurants" and sends a prompt to the generative AI model saying, "Please suggest a menu item that you would recommend when the user is in a sad mood." The generative AI model might suggest "pork bone ramen."

[1409] Prompt Sentence Examples

[1410] Suggest menu recommendations when the user is feeling sad.

[1411] In this way, the system can provide appropriate responses and product / service suggestions based on the user's emotions, provide psychological support, and increase user satisfaction.

[1412] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1413] Step 1:

[1414] The device receives the user's voice input through a microphone. The user may say, "I'm tired today, so I want to eat something delicious." This voice data becomes the input.

[1415] Step 2:

[1416] The device uses the SpeechRecognition library to convert the voice data into text, analyzes the voice data, and outputs the text "I'm tired today, so I want to eat something delicious."

[1417] Step 3:

[1418] The server sends the converted voice input to the emotion analysis API. Using the text data as input, the emotion analysis API analyzes the user's emotions. The emotion data "sad" is output as the analysis result.

[1419] Step 4:

[1420] The server generates an appropriate response based on the emotion data. Using a generative AI model, the server takes the emotion data "sad" as input and generates a prompt sentence: "Please suggest a menu item that you would recommend when the user is feeling sad." This prompt sentence is the output.

[1421] Step 5:

[1422] The server sends the generated prompt sentence to the generative AI model. Using the prompt sentence as input, the generative AI model generates an appropriate response. For example, it outputs response data such as "I recommend the pork bone ramen."

[1423] Step 6:

[1424] The server sends the generated response data to the terminal. The terminal responds to the user by voice using the response data "We recommend the pork bone ramen." The user receives the voice response "We recommend the pork bone ramen."

[1425] In this way, the system can provide appropriate responses and product / service suggestions based on the user's emotions, provide psychological support, and increase user satisfaction.

[1426] Example 2

[1427] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1428] In modern society, the number of single people and elderly people living alone is increasing, and these people often feel lonely and anxious. Furthermore, if these people are in an emergency, it is difficult for them to receive prompt and appropriate assistance. Furthermore, if the conversations provided by interactive robots are monotonous, users may become bored and stop using the robot.

[1429] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1430] In this invention, the server is a conversational robot incorporating a generative AI, a system for marketing to single people and elderly people living alone. It includes: a means for the robot to respond to everything from worries to casual everyday conversations, things to research, and purchasing desired products; a means for setting the robot's personality (age, conversational style) in advance; a means for learning the user's hobbies, preferences, way of thinking, and outlook on life through repeated conversations; a means for analyzing the user's emotional state; and a means for notifying medical institutions and local governments based on the results of the emotional analysis. This reduces the user's feelings of loneliness and mental anxiety and enables rapid response to emergencies. Furthermore, including a means for engaging the user in conversations encourages continued use of the robot.

[1431] "Generative AI" is a system that automatically generates data and information using artificial intelligence technology.

[1432] An "interactive robot" is a robot that can provide information and receive instructions through conversation with a user.

[1433] "Single person" refers to someone who lives alone.

[1434] "Elderly people living alone" refers to elderly people who live alone.

[1435] "Personality setting" refers to setting a robot's characteristics, such as its age and how it speaks, in advance.

[1436] "Hobbies and tastes" refers to the activities and interests that an individual has.

[1437] "Emotional state" refers to the type and intensity of the emotion the user is feeling.

[1438] "Sentiment analysis" refers to the process of identifying and evaluating the emotions expressed by users through their words and actions.

[1439] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[1440] "Local government" refers to a local public body responsible for local administration.

[1441] "Notification" refers to the act of communicating specific information to another institution or individual.

[1442] A "secure communication protocol" refers to a communication protocol for securely sending and receiving data.

[1443] This invention is a system that provides a variety of services to single people and elderly people living alone using an interactive robot incorporating a generation AI. Specific embodiments of this system will be described below.

[1444] 1. Program Generation

[1445] The server generates a program that contains the robot's personality and emotion engine functions, which are developed using programming languages ​​such as Python and Java.

[1446] 2. Program processing explanation

[1447] The server executes the generated program and performs the following processes.

[1448] Personality setting function

[1449] The user accesses the robot's personality setting screen using a device (such as a smartphone or PC). Here, the user selects the robot's age and conversation style. For example, it is possible to set it to speak in a youthful manner or use honorific language. The server receives the setting information sent by the user and reflects it in the robot's conversation algorithm. At this time, a natural language processing library (e.g., NLTK or spaCy) is used to adjust the conversation style.

[1450] Emotion Engine

[1451] The robot analyzes the user's emotional state through conversation with the user. For emotion analysis, it uses an emotion recognition API (e.g., Microsoft Azure's Text Analytics API). If the user continues to show sad emotions for a long period of time, the robot sends that information to a server. The server then contacts medical institutions or local governments based on the received information. This communication uses a secure communication protocol (e.g., HTTPS).

[1452] 3. Examples of concrete examples and prompts

[1453] Specific examples

[1454] When a user says to the robot, "I don't feel well today," the robot responds in a youthful voice, "What's wrong? Is something wrong?"

[1455] If the user repeatedly says, "I've been feeling sad lately," the robot analyzes this information using its emotion engine and reports it to the server, which then notifies a medical institution so that appropriate assistance can be provided.

[1456] Prompt Sentence Examples

[1457] "Please set the robot's conversation style to a younger person."

[1458] "Set it to notify medical institutions if the user expresses sad emotions."

[1459] In this way, users can enjoy conversations with robots according to their preferences, while at the same time managing their mental health appropriately.

[1460] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1461] Step 1:

[1462] The user sets the robot's personality.

[1463] The user accesses the robot's personality setting screen using a terminal, where they can select the robot's age and conversation style. For example, they can set it to speak in a youthful manner or use polite language.

[1464] Input: Configuration information entered by the user on the device (age, conversational style).

[1465] Output: The configuration information is sent to the server.

[1466] What happens: The user opens the app on their device, goes to the settings screen, selects "Speak like a young person," and turns on the "Use honorific language" option.

[1467] Step 2:

[1468] The server receives the configuration information and reflects it in the robot's conversation algorithm.

[1469] The server receives the user-submitted configuration information and adjusts the robot's conversation algorithm, changing the conversation style using natural language processing libraries (e.g., NLTK and spaCy).

[1470] Input: Configuration information submitted by the user.

[1471] Output: A tuned conversation algorithm.

[1472] What it does: The server saves the user's preferences in a database and uses a natural language processing library to change the robot's conversational style to sound more youthful.

[1473] Step 3:

[1474] The user starts a conversation with the robot.

[1475] The user starts a conversation with the robot, which reflects the user's settings. The robot then conducts the conversation based on the user's settings.

[1476] Input: What the user says.

[1477] Output: The robot's response.

[1478] Specific behavior: When the user says "Hello" to the robot, the robot responds in a youthful voice with "Yay, how are you?"

[1479] Step 4:

[1480] The robot analyzes the user's emotional state.

[1481] The robot analyzes the user's emotional state through conversation with the user, using an emotion recognition API (e.g., Microsoft Azure's Text Analytics API).

[1482] Input: What the user says.

[1483] Output: Emotion analysis results.

[1484] Specific behavior: When a user says, "I've been feeling sad lately," the robot calls the emotion recognition API and analyzes the user's emotional state as "sad."

[1485] Step 5:

[1486] The server receives the emotion analysis results and notifies medical institutions and local governments as necessary.

[1487] The server receives the emotion analysis results sent from the robot. If the user continues to show sad emotions for a long period of time, the server notifies the information to medical institutions and local governments. A secure communication protocol (e.g., HTTPS) is used.

[1488] Input: Sentiment analysis results.

[1489] Output: Notification to medical institutions and local authorities.

[1490] Specific operation: The server stores the emotion analysis results in a database, and if the "sad" emotion persists for a certain period of time, it notifies the medical institution. The server reports the user's emotional state to the medical institution using HTTPS.

[1491] (Application example 2)

[1492] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1493] In modern society, the number of single people and elderly people living alone is increasing, and these people often feel lonely and anxious. In addition, since they may not be able to respond appropriately in an emergency, there is a need to properly manage their mental health and provide prompt support when necessary. Furthermore, there is a need to provide interactive robots that users can use continuously without getting bored.

[1494] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1495] In this invention, the server is an interactive robot incorporating a generative AI, and is a system for selling to single people and elderly people living alone. It includes means for the robot to respond to everything from worries to casual everyday conversations, things you want to research, and purchasing products you want to buy, means for setting the robot's personality in advance (age, conversation style), means for learning the user's hobbies, tastes, way of thinking, and outlook on life through repeated conversations, means for monitoring the user's emotional state in real time, and means for automatically sending a notification to an emergency contact if the user feels anxiety or fear. This makes it possible to appropriately manage the user's mental health and provide prompt support when necessary.

[1496] "Generative AI" is a system that uses artificial intelligence technology to generate new information and content from data.

[1497] An "interactive robot" is a robot that can have a natural conversation with a user.

[1498] "Single person" refers to someone who lives alone.

[1499] "Elderly people living alone" refers to elderly people who live alone.

[1500] A "system" is a system in which multiple elements work together to perform a specific function.

[1501] "Personality setting" refers to setting the robot's age and conversation style in advance.

[1502] "Hobbies" refers to the things and activities that an individual likes.

[1503] "Emotional state" refers to a user's current feelings or mood.

[1504] "Real time" refers to processing or responding immediately in the current ongoing time.

[1505] "Monitoring" refers to the continuous observation of a particular condition or situation.

[1506] "Emergency Contact" refers to the contact information of a person who should be contacted in an emergency.

[1507] "Notification" refers to sending a message or signal to inform others of specific information.

[1508] This invention is a system that uses an interactive robot incorporating generative AI to appropriately manage the mental health of single people and elderly people living alone and provide prompt support as needed. Specific embodiments of this system are described below.

[1509] System Configuration

[1510] The system consists of the following main components:

[1511] 1. Conversational robots: Robots that can have natural conversations with users and incorporate generative AI.

[1512] 2. Emotion monitoring device: Using a smartphone's camera and microphone, it analyzes the user's facial expressions and tone of voice in real time.

[1513] 3. Notification system: A system that automatically sends notifications to emergency contacts if a user feels anxious or scared.

[1514] Hardware and software used

[1515] Hardware: Smartphone (camera, microphone)

[1516] Software: OpenCV (image processing library), Keras (deep learning library), Twilio (SMS sending service)

[1517] Data processing and calculation

[1518] 1. Emotion monitoring: Capture video in real time using a smartphone camera, detect faces using OpenCV, and input the detected face images into a Keras emotion recognition model to analyze the user's emotional state.

[1519] 2. Anomaly detection: If the user expresses emotions such as "Fear" or "Sad," an SMS will be sent to emergency contacts using Twilio.

[1520] 3. Personality setting: Users can set the personality of the conversational robot within the app, such as a friendly young person, a strict adult, or a polite character who uses honorific language.

[1521] Specific examples

[1522] For example, if a user feels anxious while alone at night, the emotion monitoring device will detect that emotion and the notification system will send a notification to family members or a security company, thus helping to properly manage the user's mental health and providing prompt assistance when needed.

[1523] Prompt Sentence Examples

[1524] Develop an application that automatically sends a notification to an emergency contact if the user feels anxious or scared. Emotion recognition will be performed using the smartphone camera, and emotions will be monitored in real time. If an abnormality is detected, an SMS will be sent using Twilio.

[1525] Thus, specific embodiments for carrying out the invention have been shown.

[1526] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1527] Step 1:

[1528] A user launches a smartphone application. The user's face is captured by the camera as input. The smartphone camera captures video in real time and detects the face using OpenCV. The output is an image of the detected face.

[1529] Step 2:

[1530] The server inputs the detected face image into the Keras emotion recognition model. The face image data is passed to the model as input. The server uses the emotion recognition model to analyze the facial expression and estimate the user's emotional state. The output is the emotional state (e.g., Fear, Sad, Happy, etc.).

[1531] Step 3:

[1532] The server evaluates the estimated emotional state. As input, the emotional state data is evaluated. The server determines whether the emotional state is "Fear" or "Sad." As output, an abnormal emotion flag (True / False) is obtained.

[1533] Step 4:

[1534] If the server determines that the abnormal emotion flag is true, it uses Twilio to send an SMS to the emergency contact. The abnormal emotion flag and the emergency contact information are passed as input. The server calls the Twilio API and sends a message to the emergency contact saying, "The user is experiencing anxiety or fear. Immediate action is required." The status of the SMS transmission is obtained as output.

[1535] Step 5:

[1536] The server logs the status of SMS sending. As input, the status of SMS sending is passed. The server writes the sending result to a log file for later review. As output, the log file is updated.

[1537] Step 6:

[1538] The user sets the personality of the conversational robot. The input is the personality setting selected by the user in the app (e.g., young person, strict adult, polite character using honorific language, etc.). The device applies the selected setting to the robot and changes its conversation style. The output is a conversational robot that reflects the setting.

[1539] Step 7:

[1540] The user starts a conversation with the interactive robot. The user's words are passed to the robot via a microphone as input. The device uses a generative AI to generate an appropriate response based on the user's words and responds to the user. The output is a natural continuation of the conversation.

[1541] In this way, the system monitors the user's emotional state in real time and sends notifications to emergency contacts as needed to properly manage the user's mental health.

[1542] Example 3

[1543] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1544] When using conversational robots for single people or elderly people living alone, it is necessary to properly understand the user's hobbies, tastes, and health conditions and provide personalized conversations. It is also important to adjust the conversation so that the user does not get bored and to respond quickly to emergencies. To solve these issues, a system is needed in which the robot learns through conversations with the user and responds appropriately.

[1545] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1546] In this invention, the server includes means for collecting conversation data with the user, extracting keywords using a generative AI model, and creating a user profile; means for collecting information on the user's health condition and lifestyle and reporting it to medical institutions or local governments as necessary; and means for adjusting the conversation when the user feels bored. This makes it possible to properly understand the user's hobbies, preferences, and health condition and provide personalized conversation. It also makes it possible to adjust the conversation so that the user does not become bored and to respond quickly to emergencies.

[1547] A "generative AI model" is an artificial intelligence technology that analyzes conversation data with users, extracts keywords, and creates user profiles.

[1548] An "interactive robot" is a robot that can converse with users, understand their hobbies, preferences, and health status, and respond appropriately.

[1549] A "profile" is data that compiles information such as a user's hobbies, preferences, way of thinking, and health status.

[1550] "Keywords" are important words and phrases extracted from conversations with users.

[1551] "Health status" refers to information about the user's physical condition, medical history, and daily health.

[1552] "Living conditions" refers to information about the user's daily life, such as lifestyle habits and living environment.

[1553] A "medical institution" is a facility that provides medical services, such as a hospital or clinic.

[1554] A "local government" is a government body responsible for administering a region.

[1555] A "personalized conversation" is a conversation that is individually tailored based on a user's profile.

[1556] An "emergency" is a situation in which a user requires immediate assistance, such as illness or an accident.

[1557] MODE FOR CARRYING OUT THE INVENTION

[1558] This invention is a system that uses an interactive robot incorporating a generative AI model to provide personalized conversations to single people and elderly people living alone, and monitor their health and living conditions. Specific embodiments of this system are described below.

[1559] server

[1560] The server collects conversation data sent from the robot in real time. The collected data is analyzed using a generative AI model to extract important keywords. For example, if a user says, "I went to the movies recently," the conversation data is sent to the server, which then extracts the keyword "movie."

[1561] Next, the server creates a user profile based on the extracted keywords. The profile records the user's hobbies, preferences, way of thinking, health status, etc. For example, if the keyword "movies" appears frequently, the profile will record that the user is a movie lover.

[1562] The server also collects information about the user's health and living conditions and reports it to medical institutions and local governments as necessary. For example, if a user says, "I have a headache today," that information is sent to the server, which then takes steps to contact a medical institution.

[1563] robot

[1564] The robot starts a conversation when the user speaks to it. For example, if the user says "Hello," the robot responds with "Hello, what would you like to talk about today?" The robot provides a conversation tailored to the user based on the profile information provided by the server. For example, if the user's profile indicates that they like movies, the robot will ask, "What movies have you seen recently?"

[1565] The robot also monitors the user's health condition during the conversation. For example, if the user says, "I have a headache today," the robot will suggest, "That's worrying. Should I contact a medical institution?" The robot also has the ability to adjust the conversation if it senses that the user is bored. For example, if the user says, "I'm a bit bored," the robot will change the topic by saying, "Well, how about we tell you something interesting?"

[1566] User

[1567] Users can start a conversation by talking to the robot. For example, they can say, "How is your day?" and the robot will respond. Users can share their hobbies and interests through conversation with the robot. For example, by saying, "I recently went on a trip," the robot will send that information to the server and update their profile.

[1568] If the user feels unwell, they can tell the robot. For example, if they say, "I'm not feeling well today," the robot will suggest an appropriate response. Furthermore, if an emergency occurs, the user can ask the robot for help. For example, if they say, "Help me," the robot will send the information to the server and contact a medical institution or local government.

[1569] Examples and prompts

[1570] Specific examples

[1571] 1. When users talk about travel

[1572] Robot: "Have you traveled anywhere recently?"

[1573] User: "Yes, I went to Kyoto last week."

[1574] Robot: "Kyoto is a wonderful place. What impressed you the most about it?"

[1575] 2. If the user complains of feeling unwell

[1576] Robot: "How are you feeling today?"

[1577] User: "I have a bit of a headache."

[1578] Robot: "That's concerning. May I contact medical help?"

[1579] Prompt Sentence Examples

[1580] 1. Learning about hobbies and tastes

[1581] "Identify hobbies and interests based on what they've talked about recently."

[1582] 2. Reporting on health status

[1583] "If a user complains of feeling unwell, please explain the procedure for reporting this information to a medical institution."

[1584] 3. Moderate the conversation

[1585] "Please explain how you would moderate the conversation if you sense the patron is bored."

[1586] This system enables the robot to provide more personalized services to the user and increase the user's satisfaction. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1587] Step 1:

[1588] The server collects conversation data between the robot and the user in real time.

[1589] Input: Conversation data from the robot

[1590] Output: Collected conversation data

[1591] Specific operation: For example, when a user says "hello," the robot converts the voice data into text data and sends it to the server.

[1592] Step 2:

[1593] The server extracts important keywords from the collected conversation data and uses a generative AI model to understand the context of the conversation and identify the keywords.

[1594] Input: Collected conversation data

[1595] Output: Extracted keywords

[1596] Specific operation: For example, extract the keyword "movie" from the conversation data "I went to see a movie recently."

[1597] Step 3:

[1598] The server creates a user profile based on the extracted keywords.

[1599] Input: Extracted keywords

[1600] Output: User profile

[1601] Specific operation: For example, if the keyword "movie" appears frequently, record in the user's profile that they are a movie lover.

[1602] Step 4:

[1603] The server periodically updates the profile based on new conversation data with the user.

[1604] Input: New conversation data

[1605] Output: Updated profile

[1606] Specific behavior: For example, if a user says, "I recently went on a trip," the information "I love traveling" will be added to their profile.

[1607] Step 5:

[1608] The robot will start a conversation when the user speaks to it.

[1609] Input: Voice input from the user

[1610] Output: Robot response

[1611] Specific behavior: For example, when the user says "Hello," the robot responds, "Hello, what would you like to talk about today?"

[1612] Step 6:

[1613] The robot provides conversation tailored to the user based on the profile information provided by the server.

[1614] Input: Profile information from the server

[1615] Output: Personalized conversation

[1616] Specific behavior: For example, if the user's profile records that they like movies, the robot will ask, "What movies have you seen recently?"

[1617] Step 7:

[1618] The robot monitors the user's health status through conversation.

[1619] Input: User's health-related comments

[1620] Output: Health monitoring results

[1621] Specific behavior: For example, if a user says, "I have a headache today," the robot will suggest, "That's worrying. Should I contact a medical institution?"

[1622] Step 8:

[1623] The robot will adjust the conversation if it senses that the user is bored.

[1624] Input: User's comment about boredom

[1625] Output: Adjusted conversation

[1626] Specific behavior: For example, if the user says, "This is a bit boring," the robot will change the topic by saying, "Well, how about we tell you something interesting?"

[1627] Step 9:

[1628] If the user experiences an emergency, the robot will send the information to a server and contact medical institutions or local authorities.

[1629] Input: User's statement regarding the emergency

[1630] Output: Sending emergency information to the server and contacting medical institutions and local authorities

[1631] Specific behavior: For example, if a user says "help me," the robot immediately sends information to the server and prompts the appropriate response.

[1632] (Application example 3)

[1633] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1634] There is a need to solve problems faced by single people and elderly people living alone, such as loneliness, delayed response in emergencies, and boredom in daily life. There is also a need for a system that can provide conversations tailored to the user's hobbies and ways of thinking, and that users can use for a long time without getting bored. Furthermore, there is a need for a system that can grasp the user's health condition and lifestyle patterns, and respond quickly when abnormalities occur.

[1635] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1636] In this invention, the server is a conversational robot incorporating generative AI, a system for marketing to single people and elderly people living alone. It includes: a means for the robot to respond to everything from worries to casual everyday conversations, things to research, and purchasing desired products; a means for setting the robot's personality (age, conversational style) in advance; a means for learning the user's hobbies, preferences, way of thinking, and outlook on life through repeated conversations; a means for learning the user's lifestyle patterns and health status, and contacting a medical institution or local government if abnormal behavior or poor health is detected; and a means for adjusting the conversation if the user feels bored. This reduces the user's sense of loneliness and enables rapid response in emergencies. Furthermore, by keeping the user interested, the system can be used for long periods of time.

[1637] "Generative AI" is a system that uses artificial intelligence technology to generate data such as text and voice.

[1638] An "interactive robot" is a robot that can have a natural conversation with a user.

[1639] "Single person" refers to someone who lives alone.

[1640] "Elderly people living alone" refers to elderly people who live alone.

[1641] A "system" is a set of mechanisms in which multiple elements function in conjunction with one another.

[1642] "Problems" refer to problems or difficult situations that users face.

[1643] "Daily conversation" refers to general conversation in everyday life.

[1644] "What you want to find out" refers to the information or data that the user wants to know.

[1645] The "item you want to buy" refers to an item that you want to purchase.

[1646] "Personality setting" refers to setting a robot's characteristics, such as its age and how it speaks, in advance.

[1647] "Hobbies and interests" refers to the activities and areas of interest that users prefer.

[1648] "Mindset" refers to the user's thought patterns and values.

[1649] "Life perspective" refers to the user's perspective and feelings about life.

[1650] "Lifestyle patterns" refer to the user's daily habits and behavioral tendencies.

[1651] "Health Status" refers to the state of a user's physical and mental health.

[1652] "Abnormal behavior" refers to actions or reactions that are different from the norm.

[1653] "Illness" refers to physical discomfort or signs of illness felt by the user.

[1654] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[1655] "Local government" refers to a local public body responsible for local administration.

[1656] "To communicate" refers to the use of communication means to convey information.

[1657] "Boredom" refers to a state in which time seems to drag on without interest or stimulation.

[1658] "Adjusting the conversation" means changing the topic and speaking style to suit the user's interests and mood.

[1659] The system for implementing this invention uses an interactive robot incorporating generative AI. The system is marketed to single people and elderly people living alone, and provides various functions through conversations with the user. Specific embodiments of the system are described below.

[1660] System configuration

[1661] The system consists of the following main components:

[1662] 1. Conversational robot: A robot that can have natural conversations with users.

[1663] 2. Generative AI model: An artificial intelligence technology that analyzes user conversations and generates appropriate responses.

[1664] 3. Server: This is the central system that processes and stores data.

[1665] 4. Communication means: A network for data communication between the robot and the server.

[1666] Program processing

[1667] The server analyzes the conversation with the user using the generative AI model and executes the following measures:

[1668] 1. Creating a user profile: Learn about the user's hobbies, preferences, way of thinking, and outlook on life from conversation history and create a profile.

[1669] 2. Anomaly detection and notification: Learns the user's lifestyle patterns and health condition, and contacts medical institutions or local governments if it detects abnormal behavior or poor health.

[1670] 3. Conversation adjustment: If you sense that the user is bored, adjust the conversation to keep them interested.

[1671] Hardware and software used

[1672] Hardware: Smartphones, interactive robots

[1673] Software: Python, OpenAI API, requests library

[1674] Data processing and calculation

[1675] The server collects conversation history with the user and performs the following data processing and calculations using a generative AI model.

[1676] Analysis of conversation history: Extract keywords from users' comments and create a profile.

[1677] Anomaly detection: Analyzes user behavior patterns and notifies you if an anomaly is detected.

[1678] Conversation generation: Generates appropriate conversation content based on the user's profile and current mood.

[1679] Specific examples

[1680] For example, if a user says, "I haven't been feeling well lately," the server will analyze that information and contact a medical institution. If a user says, "I'm bored," the server will provide new topics of conversation, such as, "What interesting movies have you seen recently?"

[1681] Prompt Sentence Examples

[1682] Below is an example of a prompt sentence to input to the generative AI model.

[1683] Create a user profile using the following conversation history:

[1684] "I've been wanting to go on a trip lately."

[1685] "What kind of place do you like?"

[1686] "I like places with a lot of nature."

[1687] User Profile:

[1688] In this way, the system can support users' daily lives and respond quickly in emergencies.

[1689] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1690] Step 1:

[1691] The server collects conversation histories with users.

[1692] Input: User statement

[1693] Output: Conversation history data

[1694] Specific operation: When a user speaks to the interactive robot, their speech is converted into text data using speech recognition technology and sent to the server, which then saves this text data as a conversation history.

[1695] Step 2:

[1696] The server extracts keywords from the conversation history and creates a user profile using a generative AI model.

[1697] Input: Conversation history data

[1698] Output: User profile

[1699] How it works: The server analyzes conversation history data and extracts important keywords. It then inputs prompts into a generative AI model to generate a profile that reflects the user's interests, tastes, and way of thinking.

[1700] Step 3:

[1701] The server learns the user's lifestyle patterns and health status, and detects abnormal behavior or poor health.

[1702] Input: User profile, conversation history data

[1703] Output: Anomaly detection results

[1704] How it works: The server models the user's lifestyle patterns and health status based on their profile and conversation history. If abnormal behavior or poor health is detected, it generates an anomaly detection result.

[1705] Step 4:

[1706] The server contacts medical institutions and local governments based on the abnormality detection results.

[1707] Input: Anomaly detection results

[1708] Output: Information message

[1709] Specific operation: When an anomaly is detected, the server sends a notification message to pre-defined contacts (medical institutions or local governments) containing the user's profile and details of the anomaly.

[1710] Step 5:

[1711] The server moderates the conversation if it senses that the user is bored.

[1712] Input: User profile, current mood

[1713] Output: Adjusted conversation

[1714] How it works: Based on the user's profile and current mood, the server inputs prompts into the generative AI model to generate new, interesting conversation content, which is then provided to the user through the conversational robot.

[1715] Step 6:

[1716] The server provides the generated conversation content to the user to continue the conversation.

[1717] Input: Adjusted conversation

[1718] Output: User statement

[1719] Specific operation: The server sends the generated conversation content to the interactive robot, which then verbally conveys it to the user. When the user makes a new comment, that comment is sent back to the server and added to the conversation history.

[1720] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1721] The data generation model 58 is a so-called generative AI (Artificial Intelligence).

[1722] An example of a data generation model58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1723] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1724] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1725] [Third embodiment]

[1726] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1727] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1728] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1729] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1730] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1731] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1732] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1733] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1734] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1735] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1736] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1737] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1738] "Example 1"

[1739] One aspect of the present invention is a conversational robot incorporating a generative AI. This robot is marketed to single people and elderly people living alone, and can handle everything from worries to everyday conversations, things to look up, and even purchasing products. Specifically, the robot has a voice recognition function and responds by analyzing voice input from the user. It can also purchase products and search for information via its internet connection.

[1740] "Example 2"

[1741] Furthermore, the robot's personality can be set in advance. Specifically, users can set the robot's age and conversation style. For example, they can set the robot to speak like a young person or to use honorific language. This allows users to enjoy conversations with the robot according to their preferences.

[1742] "Example 3"

[1743] Furthermore, through repeated conversations, the robot learns about the user's hobbies, preferences, way of thinking, and outlook on life. Specifically, the robot extracts keywords from conversations with the user and analyzes them to create a profile of the user. For example, if the user often talks about travel, the robot will learn that the user loves traveling. This allows the robot to provide conversations tailored to the user.

[1744] "Example 4"

[1745] Furthermore, the robot will contact medical institutions and local governments based on the content of the conversation to respond to cases of lonely deaths and emergencies. Specifically, the robot will collect information about the user's health and living situation and report it to medical institutions and local governments as necessary. For example, if the user complains of feeling unwell, the robot will contact a medical institution with that information, enabling early response.

[1746] The processing flow of each embodiment will be described below.

[1747] "Example 1"

[1748] Step 1: An interactive robot incorporating generative AI is launched.

[1749] Step 2: The robot uses its voice recognition function to analyze the voice input from the user.

[1750] Step 3: Based on the analysis results, the robot generates an appropriate response and outputs it to the user.

[1751] Step 4: If the user requests to purchase a product or search for information, the robot will perform the corresponding operation via its internet connection function.

[1752] "Example 2"

[1753] Step 1: The user sets the robot's personality from the settings screen.

[1754] Step 2: The user sets the robot's age and conversation style.

[1755] Step 3: Once the setup is complete, the robot will conduct a conversation based on the settings.

[1756] "Example 3"

[1757] Step 1: The robot extracts keywords from the conversation with the user.

[1758] Step 2: Analyze the extracted keywords and create a user profile.

[1759] Step 3: Based on the profile you create, the robot will provide you with a tailored conversation.

[1760] "Example 4"

[1761] Step 1: The robot collects information about the user's health and living conditions.

[1762] Step 2: If the collected information exceeds a certain threshold, the robot reports it to a medical institution or local government.

[1763] Step 3: The medical institution or local government that receives the report will take appropriate action for the user.

[1764] Example 1

[1765] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1766] In modern society, the number of single people and elderly people living alone is increasing, and these people often feel lonely and anxious. In addition, various support is needed in daily life, such as searching for information and purchasing products, but it is difficult for them to do these things alone. Furthermore, due to the lack of means to respond quickly in emergencies, lonely deaths and health problems have become serious issues.

[1767] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1768] In this invention, the server includes a means for converting voice input into text data using a voice recognition function, a means for generating appropriate responses from the text data using a generative AI model, and a means for searching for information and purchasing products using an internet connection function. This allows users to consult with the robot about their worries, engage in everyday conversations, search for information, and purchase products simply by speaking to the robot. The robot can also learn the user's preferences and way of thinking through conversations and provide more personalized responses. Furthermore, in emergencies, the robot can cooperate with medical institutions and local governments to respond quickly.

[1769] "Generative AI" is a system that uses artificial intelligence technology to generate natural language responses from text data.

[1770] An "interactive robot" is a robot that converses with users by combining technologies such as voice recognition, generative AI, and voice synthesis.

[1771] "Single people / elderly people living alone" refers to people who live alone, especially elderly people.

[1772] "Voice recognition function" is a technology that converts voice input into text data.

[1773] A "generative AI model" is an artificial intelligence model that generates appropriate responses based on input text data.

[1774] "Internet connectivity" means the ability to access external information sources and services via the Internet.

[1775] The "voice synthesis function" is a technology that converts text data into voice and outputs it.

[1776] "Personality settings" is a function that allows you to pre-set the robot's characteristics, such as its age and conversational style.

[1777] "Hobbies and tastes" refers to the user's preferences and interests.

[1778] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[1779] "Local government" refers to a government agency responsible for local government administration.

[1780] An "emergency" refers to an urgent situation faced by a user, such as a health problem or accident.

[1781] This invention is a system that provides multifunctional support to single people and elderly people living alone using an interactive robot incorporating generative AI. This system is realized by combining technologies such as voice recognition, generative AI, internet connection, and voice synthesis.

[1782] Hardware and software used

[1783] Speech recognition functionality: Use a speech recognition API (e.g., Google Cloud Speech-to-Text API) to convert voice input into text data.

[1784] Generative AI models: Use generative AI models (e.g., OpenAI's GPT-4) to generate appropriate responses from text data.

[1785] Internet connection functions: Use the Internet connection functions to search for information and purchase products. Specifically, use the Google Search API and Amazon API.

[1786] Speech synthesis function: Use a speech synthesis API (e.g., Google Cloud Text-to-Speech API) to convert text data into speech and output it.

[1787] Specific operation of the system

[1788] When a user speaks to the robot, it captures the voice through its built-in microphone. It then uses a speech recognition API to convert the voice data into text data. This text data is sent to the server and input into the generative AI model. The generative AI model generates an appropriate response based on the input text data and returns the response to the server.

[1789] The server receives the response from the generative AI model and, if necessary, uses its internet connection to retrieve additional information. For example, if a user asks, "What restaurants are nearby?", the server uses the Google Search API to retrieve information about nearby restaurants. The retrieved information is then combined with the response from the generative AI model to generate the final response.

[1790] The final response is sent from the server to the device, which then uses a speech synthesis API to convert the text data into speech, which is then spoken to the user.

[1791] Specific examples

[1792] Example 1: Everyday conversation

[1793] User: "How's the weather today?"

[1794] Robot: "It's sunny today and the temperature is 25 degrees."

[1795] Example 2: Information search

[1796] User: "What restaurants are nearby?"

[1797] Robot: "There's an Italian restaurant nearby. It has a 4.5 rating."

[1798] Example 3: Purchasing a product

[1799] User: "I want to buy a new smartphone."

[1800] Robot: "The latest smartphone is on sale in the online store. Would you like to buy it?"

[1801] Prompt Sentence Examples

[1802] "When a user speaks to the robot, it uses speech recognition to convert the speech into text, which is then fed into a generative AI model to generate a response. For example, if a user asks, 'What's the weather like today?' the robot will respond, 'It's sunny today, and the temperature is 25 degrees.'"

[1803] In this way, the system of the present invention can meet a variety of user needs by combining voice recognition, generation AI, internet connection, and voice synthesis.

[1804] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1805] Step 1:

[1806] The user provides voice input.

[1807] The user speaks to the robot, for example, asking, "What's the weather like today?" The input is the user's voice, and the output is the audio data captured by the robot's microphone.

[1808] Step 2:

[1809] The device converts the speech to text.

[1810] The device captures the user's voice through the built-in microphone. Then, it uses a speech recognition API to convert the voice data into text data. The input is the voice data, and the output is the text data "What's the weather like today?"

[1811] Step 3:

[1812] The terminal transmits the text data to the server.

[1813] The terminal sends the converted text data to the server. The input is the text data, and the output is the text data sent to the server.

[1814] Step 4:

[1815] The server sends the text data to the generative AI model.

[1816] The server inputs the received text data into the generative AI model. The input is the text data "What's the weather like today?", and the output is the prompt input into the generative AI model.

[1817] Step 5:

[1818] The server receives the response from the generative AI model.

[1819] The generative AI model generates an appropriate response based on the input text data. The server receives this response. The input is the prompt entered into the generative AI model, and the output is the response text: "Today is sunny and the temperature is 25 degrees."

[1820] Step 6:

[1821] The server performs internet searches and purchases as needed.

[1822] When a user requests information search or product purchase, the server uses its Internet connection to obtain the necessary information. For example, it uses the Google Search API to obtain weather information. The input is text data based on the user's request, and the output is the obtained information.

[1823] Step 7:

[1824] The server sends a final response to the terminal.

[1825] The server compiles the responses from the generative AI model and the information obtained from the internet to generate a final response, which is then sent to the terminal. The input is the response text from the generative AI model and the obtained information, and the output is the final response text sent to the terminal.

[1826] Step 8:

[1827] The terminal responds to the user verbally.

[1828] The device converts the final response received from the server into speech using a speech synthesis API. It responds to the user via voice. The input is the final response text, and the output is a voice response to the user. For example, it might respond, "Today is sunny, and the temperature is 25 degrees."

[1829] (Application example 1)

[1830] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1831] Single people and elderly people living alone often feel lonely and anxious in their daily lives, and require support, especially when it comes to ordering meals and managing their health. However, there are only a limited number of effective systems to address these needs. There is also a lack of systems that can provide appropriate suggestions based on the user's hobbies, tastes, and health status. This creates a challenge for users, making it difficult for them to live their lives with peace of mind.

[1832] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1833] In this invention, the server is an interactive robot incorporating a generative AI, a system for selling to single people and elderly people living alone, and includes: means for the robot to respond to everything from worries to casual everyday conversations, things you want to research, and purchasing products you want; means for setting the robot's personality in advance (age, conversational style); means for learning the user's hobbies, preferences, way of thinking, and outlook on life through repeated conversations; means for analyzing voice input and ordering food; means for connecting to a food delivery service via the Internet and completing the order; and means for storing the user's food preferences and allergy information and suggesting appropriate menus. This helps reduce the user's sense of loneliness and anxiety, allowing them to live their lives with peace of mind.

[1834] "Generative AI" refers to algorithms and models that use artificial intelligence techniques to generate new data and information.

[1835] An "interactive robot" is a robot that can converse with humans using voice recognition and natural language processing technology.

[1836] A "single person" is someone who lives alone.

[1837] "Elderly people living alone" refers to elderly people who live alone.

[1838] "Personality setting" refers to setting a robot's characteristics, such as its age and how it speaks, in advance.

[1839] Hobbies and interests are the activities and things that an individual likes.

[1840] "Audio input" refers to obtaining audio as digital data through a device such as a microphone.

[1841] "Ordering food" refers to the act of requesting food from a restaurant or food delivery service.

[1842] "Internet connection" means access to the Internet through a network.

[1843] "Food delivery service" means a service that delivers ordered meals to a specified location.

[1844] "Allergy information" refers to information that indicates an allergic reaction to a specific food ingredient or substance.

[1845] "Menu suggestion" refers to recommending an appropriate meal menu based on the user's preferences and health condition.

[1846] The system for implementing this invention uses an interactive robot incorporating generative AI. The system is marketed to single people and elderly people living alone, and can handle everything from worries to everyday conversations, things you want to research, and even purchasing products you want to buy. Specific embodiments of the system are described below.

[1847] Hardware and Software Configuration

[1848] The system uses the following hardware and software:

[1849] Hardware: Microphone (for voice input), internet-enabled robot

[1850] Software: Python, SpeechRecognition library, OpenAI API, Requests library

[1851] Data processing and calculation

[1852] 1. Acquiring voice input:

[1853] The user provides voice input through a microphone.

[1854] Use the speech recognition library (SpeechRecognition) to convert speech into text data.

[1855] 2. Use of generative AI models:

[1856] The converted text data is sent as a prompt to the generative AI model (OpenAI API).

[1857] The AI ​​model generates an appropriate response based on the prompt.

[1858] 3. Food delivery orders:

[1859] Connect to the API of a food delivery service based on the user's order.

[1860] Use the Requests library to send the order data to the food delivery service and complete the order.

[1861] 4. User information storage and suggestions:

[1862] The database stores the user's dietary preferences and allergy information.

[1863] The next time you order, the system will suggest an appropriate menu item based on the stored information.

[1864] Specific examples

[1865] For example, if a user says, "I want to order a pizza," the system operates as follows:

[1866] 1. Acquiring voice input:

[1867] The user speaks into the microphone, "I'd like to order a pizza."

[1868] The speech recognition library converts this speech into text, generating the text data "I would like to order a pizza."

[1869] 2. Use of generative AI models:

[1870] The generated text data is sent to the OpenAI API as a prompt.

[1871] Example prompt: "The meal the user would like to order is: Pizza."

[1872] The AI ​​model generates the response, "What kind of pizza would you like?"

[1873] 3. Food delivery orders:

[1874] The user responds, "Margherita pizza."

[1875] The system sends this information to the food delivery service's API to complete the order.

[1876] 4. User information storage and suggestions:

[1877] The database stores that the user likes Margherita pizza.

[1878] From the next time onwards, the user will be suggested a Margherita pizza the next time they order pizza.

[1879] In this way, the system can help single people and elderly people living alone to easily order meals, improving the quality of their daily lives.

[1880] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1881] Step 1:

[1882] The user provides voice input through a microphone. When the user says, "I'd like to order a pizza," the microphone captures the voice. The input is the voice data, and the output is the captured voice data.

[1883] Step 2:

[1884] The device uses a speech recognition library (SpeechRecognition) to convert the captured voice data into text data. Specifically, the speech recognition library analyzes the voice data and generates the text data "I would like to order a pizza." The input is voice data, and the output is text data.

[1885] Step 3:

[1886] The device sends the generated text data as a prompt to the generative AI model (OpenAI API). Specifically, the text data "I would like to order pizza" is sent as a prompt to the AI ​​model, which generates an appropriate response. The input is text data, and the output is the response text generated by the AI ​​model.

[1887] Step 4:

[1888] The server receives the response from the generative AI model and responds to the user by voice, asking, "What kind of pizza would you like?" Specifically, the generated response text is converted into voice data using speech synthesis technology and transmitted to the user through a speaker. The input is the response text, and the output is voice data.

[1889] Step 5:

[1890] The user again inputs voice through the microphone and answers "Margherita pizza." The microphone captures the voice and sends it to the device as voice data. The input is voice data, and the output is the captured voice data.

[1891] Step 6:

[1892] The device again uses the speech recognition library to convert the captured voice data into text data. Specifically, the speech recognition library analyzes the voice data and generates the text data "Margherita pizza." The input is voice data, and the output is text data.

[1893] Step 7:

[1894] The server sends the text data "Margherita Pizza" to the food delivery service's API to complete the order. Specifically, it uses the Requests library to send the text data as a POST request to the food delivery service's API endpoint. The input is the text data, and the output is a confirmation of the order completion from the food delivery service.

[1895] Step 8:

[1896] The server stores the user's food preferences and allergy information in a database. Specifically, the user's preference for "Margherita pizza" is saved in the database and used for future orders. The input is the user's preferences and allergy information, and the output is the information saved in the database.

[1897] Step 9:

[1898] From the next time onwards, when the user orders a meal again, the server retrieves the user's preferences and allergy information from the database and suggests an appropriate menu. Specifically, based on the information retrieved from the database, the server suggests to the user, "How about a Margherita pizza?" The input is the information retrieved from the database, and the output is the suggested text.

[1899] Example 2

[1900] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1901] In modern society, the number of single people and elderly people living alone is increasing, and these people often feel lonely. Furthermore, they need various support in their daily lives, such as helping with worries, researching things they want to know, and purchasing products they want to buy, but it is difficult to solve these problems on their own. Furthermore, they are required to learn the user's hobbies, tastes, and ways of thinking, and to provide continuously interesting conversations. Conversational robots are effective in solving these problems, but current systems are not able to adequately address them.

[1902] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1903] In this invention, the server includes means for a user to set the robot's personality using a terminal, means for the terminal to send the setting data to the server, means for the server to send a prompt sentence to the generative AI model, means for the generative AI model to generate a response, means for the server to send the generated response to the robot, and means for the robot to converse with the user, thereby enabling the user to enjoy a conversation with the robot according to their preferences.

[1904] A "generative AI model" is a type of artificial intelligence that generates text data using natural language processing technology.

[1905] A "prompt sentence" is an input sentence that prompts a generative AI model to generate a specific response.

[1906] An "interactive robot" is a robot that can converse with a user through voice or text.

[1907] A "terminal" is a device operated by a user, and includes smartphones, personal computers, etc.

[1908] A "server" is a computer system that processes and stores data over a network.

[1909] "Personality settings" is a feature that allows users to customize the robot's age and conversation style.

[1910] A "response" is text data that a generative AI model generates based on a prompt sentence.

[1911] A "user" is a person who operates an interactive robot and has a conversation with it.

[1912] "Setting data" is information about the robot's personality settings that the user inputs through the terminal.

[1913] A "conversation" is a communication exchange between a user and an interactive robot.

[1914] This invention is a system that uses an interactive robot incorporating a generative AI model to help single people and elderly people living alone reduce their feelings of loneliness and receive support in their daily lives. Specific embodiments of this system are described below.

[1915] Hardware and software used

[1916] Hardware: Robot body, server, user terminal (PC, smartphone, etc.)

[1917] Software: Generative AI models (e.g., models using natural language processing techniques), robot control software, user interface (UI) software

[1918] Data processing and calculation

[1919] 1. The user sets the robot's personality using the device

[1920] Users use the device's UI to set the robot's age and how it will converse.

[1921] Example: A user opens an app on their smartphone and selects "Age: Young" and "Conversational style: Casual" on the settings screen.

[1922] 2. The device sends the configuration data to the server.

[1923] The terminal transmits the setting data entered by the user to the server.

[1924] Example: The device packages configuration data in JSON format and sends it to the server using the HTTPS protocol.

[1925] 3. The server sends a prompt to the generative AI model

[1926] The server sends a prompt to the generative AI model based on the received configuration data.

[1927] Example: The server generates a prompt saying, "Generate a response from a robot that can have casual conversations like a young person," and sends it to the API endpoint of the generative AI model.

[1928] 4. The generative AI model generates a response

[1929] The generative AI model generates the robot's response based on the prompt sentence.

[1930] Example: A generative AI model parses a prompt and generates the response "Hey there! What are you doing today?"

[1931] 5. The server generates a response and sends it to the robot

[1932] The server sends the response received from the generative AI model to the robot.

[1933] Example: The server sends the response data to the robot control software, which outputs it as sound from the robot's speaker.

[1934] 6. The robot will converse with the user

[1935] The robot will then converse with the user based on the received response.

[1936] Example: A robot speaks to the user, "Hey there! What are you doing today?" When the user responds, the robot sends the data back to the server and the process repeats to generate the next response.

[1937] Prompt Sentence Examples

[1938] "Generate responses from a robot that can have casual conversations like a young person."

[1939] "Generate a polite robotic response using honorific language."

[1940] This system allows users to enjoy conversations with robots according to their preferences, reducing feelings of loneliness and providing support for daily life.

[1941] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1942] Step 1:

[1943] The user uses the terminal to set the robot's personality.

[1944] Input: Configuration data that users enter through the device's UI (e.g., age, speech patterns).

[1945] Specific actions: The user opens the app on their smartphone and selects "Age: Young" and "Conversation style: Casual" on the settings screen.

[1946] Output: The configuration data is saved to the device.

[1947] Step 2:

[1948] The terminal sends the setting data to the server.

[1949] Input: Configuration data entered by the user.

[1950] What happens: The device packages the configuration data in JSON format and sends it to the server using the HTTPS protocol.

[1951] Output: The configuration data is sent to the server.

[1952] Step 3:

[1953] The server sends a prompt to the generative AI model.

[1954] Input: The configuration data received by the server.

[1955] Specific operation: The server generates a prompt sentence, "Please generate a response from a robot that can have casual conversation like a young person," and sends it to the API endpoint of the generative AI model.

[1956] Output: The prompt is sent to the generative AI model.

[1957] Step 4:

[1958] A generative AI model generates a response.

[1959] Input: The prompt received by the generative AI model.

[1960] What it does: The generative AI model analyzes the prompt and generates the response, "Hey there! What are you doing today?"

[1961] Output: The generated response is sent back to the server.

[1962] Step 5:

[1963] The server generates a response and sends it to the robot.

[1964] Input: The response returned by the generative AI model.

[1965] Specific operation: The server sends the response data to the robot control software, which outputs it as voice from the robot's speaker.

[1966] Output: The response data is sent to the robot.

[1967] Step 6:

[1968] The robot converses with the user.

[1969] Input: The response data received by the robot.

[1970] What it does: The robot asks the user, "Hey! What are you doing today?" When the user responds, the robot sends the data back to the server and the process repeats, generating the next response.

[1971] Output: The conversation with the user continues.

[1972] (Application example 2)

[1973] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1974] In modern society, the number of single people and elderly people living alone is increasing, and supporting their loneliness and daily life is becoming an important issue. Another problem is the lack of people to talk to to alleviate the worries and anxieties they experience on a daily basis. Furthermore, there is a need to realize more friendly and timeless communication by providing conversations tailored to the user's preferences.

[1975] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server is an interactive robot incorporating a generative AI, and is a system for selling to single people and elderly people living alone. The system includes: means for the robot to respond to everything from worries to casual everyday conversations, things to research, and purchasing products; means for enabling the robot's personality to be set in advance (age, conversational style); means for learning the user's hobbies, tastes, way of thinking, and outlook on life through repeated conversations; means for the user to set the robot's personality using a smart device; means for generating natural conversations using a generative AI model based on the set personality; and means for the robot to speak the generated conversation content to the user. This allows for dialogue tailored to the user's preferences, reducing feelings of loneliness and supporting daily life.

[1976] "Generative AI" is a system that uses artificial intelligence technology to process natural language and generate dialogue with users.

[1977] An "interactive robot" is a robot that can communicate with users through voice and text.

[1978] "Single person" refers to someone who lives alone.

[1979] "Elderly people living alone" refers to elderly people who live alone.

[1980] A "system" is a structure in which multiple elements are combined to achieve a specific function.

[1981] "Worries" refer to problems and anxieties that users experience in their daily lives.

[1982] "Daily conversation" refers to general conversation in everyday life.

[1983] "What you want to know" refers to the information or questions that users want to know.

[1984] "Products you want to buy" refers to items that users are considering purchasing.

[1985] "Personality settings" refers to the user pre-setting the robot's age and conversation style.

[1986] "Hobbies and tastes" refers to the user's interests and preferences.

[1987] "Thoughts" refers to the user's thoughts and opinions.

[1988] "Life philosophy" refers to the user's perspective and values ​​on life.

[1989] "Smart devices" refer to electronic devices that can connect to the Internet, such as smartphones and tablets.

[1990] "Generative AI model" refers to the algorithms and data models used to realize generative AI.

[1991] "Natural conversation" refers to conversations that are generated in a manner that is close to dialogue between humans.

[1992] The system for implementing this invention uses an interactive robot incorporating generative AI. The system is marketed to single people and elderly people living alone, and can handle everything from worries to everyday conversations, research needs, and product purchases. A specific embodiment of this system is described below.

[1993] System configuration

[1994] The system consists of the following main components:

[1995] 1. Conversational robot: A robot that communicates with users through voice and text.

[1996] 2. Smart Device: A device such as a smartphone or tablet that allows the user to set the robot's personality.

[1997] 3. Generative AI models: Algorithms and data models that perform natural language processing and generate user interactions.

[1998] Program processing

[1999] The server implements the system using the following means:

[2000] 1. Personality setting method: The user uses a smart device to set the robot's age and conversation style. For example, if the user sets the robot to "speak casually like a young person," the prompt text will be as follows:

[2001] Speak casually, like you're a young person.

[2002] 2. Conversation generation method: A generative AI model is used based on the set personality to generate natural conversations. The generative AI model used is OpenAI's GPT-3.

[2003] 3. Conversation output means: The robot speaks the generated conversation content to the user, allowing the user to enjoy interacting with the robot.

[2004] Specific examples

[2005] For example, when a user sets a robot's personality using a smartphone, the procedure is as follows:

[2006] 1. The user opens the smartphone application and sets the robot's age and conversation style.

[2007] 2. The configuration information is sent to the server and a prompt is generated.

[2008] 3. Based on the prompt, the generative AI model generates natural conversation.

[2009] 4. The generated conversation content is sent to the robot, which then speaks to the user.

[2010] In this way, the system can provide interactions tailored to the user's preferences, reducing feelings of loneliness and supporting daily life.

[2011] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2012] Step 1:

[2013] The user configures the robot's personality using a smart device. Specifically, the user opens an application on their smartphone or tablet and configures the robot's age and conversation style. The input includes the user's selection of age and conversation style. The output is the configuration information.

[2014] Step 2:

[2015] The terminal sends the user's setting information to the server. Specifically, the setting information is sent from the smart device to the server. The input includes the user's age and conversation style. The server receives the setting information as output.

[2016] Step 3:

[2017] The server generates a prompt sentence based on the configuration information it receives. Specifically, it analyzes the configuration information and creates a prompt sentence suitable for the generative AI model. The input includes the user's configuration information. The output is the generated prompt sentence. For example, a generated prompt sentence might be, "Please speak casually, like a young person."

[2018] Step 4:

[2019] The server uses a generative AI model to generate natural conversations. Specifically, the generated prompt sentence is input into a generative AI model (e.g., OpenAI's GPT-3) to generate the conversation content. The input includes the prompt sentence. The output is the generated conversation content.

[2020] Step 5:

[2021] The server sends the generated conversation content to the robot. Specifically, the server sends the generated conversation content to the interactive robot, and the robot prepares to speak the content to the user. The generated conversation content is included as input. The robot receives the conversation content as output.

[2022] Step 6:

[2023] The robot speaks the generated conversation content to the user. Specifically, the robot uses speech synthesis technology to convey the generated conversation content to the user by voice. The generated conversation content is included as input. The output allows the user to enjoy a dialogue with the robot.

[2024] Example 3

[2025] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2026] There is a need to reduce loneliness among single people and elderly people living alone and to provide support in their daily lives. It is also important to monitor the user's health and living situation and respond quickly to emergencies. Conventional interactive robots have had difficulty fully understanding the user's hobbies and health and responding appropriately. Another challenge is maintaining a conversation so that the user does not get bored.

[2027] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[2028] In this invention, the server includes a means for collecting conversations with the user as voice input, a means for converting the collected voice data into text data, and a means for extracting keywords from the converted text data, thereby enabling the server to understand the user's hobbies and health condition and provide appropriate support.

[2029] "Generative AI" is a system that uses artificial intelligence technology to generate data such as text and voice.

[2030] An "interactive robot" is a robot that can converse with users through voice or text.

[2031] "Single people / elderly people living alone" refers to people who live alone, and specifically includes elderly people.

[2032] "Personality setting" refers to setting a robot's characteristics, such as its age and how it speaks, in advance.

[2033] "Hobbies and interests" refers to the activities and areas of interest that users prefer.

[2034] A "profile" is data that compiles information such as a user's hobbies, thoughts, and health status.

[2035] "Voice input" refers to collecting a user's speech as voice data.

[2036] "Text data" refers to data obtained by converting voice data into character information.

[2037] "Keywords" are important words and phrases extracted from conversations.

[2038] "Generative AI model" refers to an algorithm or system that uses artificial intelligence technology to generate text or speech.

[2039] "Audio output" refers to the presentation of generated text or information to the user as audio.

[2040] "Health status" refers to information about the user's physical condition and medical care.

[2041] "Living situation" refers to information about the user's daily life.

[2042] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[2043] "Local government" refers to a local administrative body that plays a role in supporting the lives of residents.

[2044] This invention is a system that uses an interactive robot incorporating a generative AI to help single people and elderly people living alone reduce feelings of loneliness and provide support for their daily lives. Specific embodiments of this system are described below.

[2045] Hardware and software used

[2046] Hardware: Interactive robots (e.g., general robotic platforms)

[2047] Software: Natural language processing engines (e.g., general natural language processing APIs), generative AI models (e.g., general generative AI models)

[2048] Explanation of program processing

[2049] Conversation data collection

[2050] The terminal (interactive robot) collects conversations with the user as voice input. Specifically, when the user speaks to the robot, the robot captures the voice with a microphone and saves it as voice data.

[2051] Keyword extraction

[2052] The device uses speech recognition software to convert the collected voice data into text data. The server then uses a natural language processing engine to extract keywords from the converted text data. For example, if the voice data is converted into text such as "I want to go on a trip recently," keywords such as "travel" and "want to go" are extracted.

[2053] Creating a profile

[2054] The server analyzes the extracted keywords and creates a user profile. For example, if the keyword "travel" appears frequently, the server records the user as "travel lover" in the user's profile.

[2055] Conversational Offerings

[2056] The server uses the generative AI model to generate a conversation based on the user's profile. The device then provides the generated conversation to the user as voice output. For example, the server generates a question such as, "Is there anywhere you'd like to travel to recently?", and the device then verbally replies the question to the user.

[2057] Health monitoring

[2058] The device collects information about the user's health and lifestyle during the conversation. The server analyzes the collected information and checks for any abnormalities. For example, if a user says, "I haven't been feeling well lately," the device sends that information to the server, which then analyzes it.

[2059] Emergency response

[2060] The server will report the results of the health status analysis to medical institutions or local governments as necessary. For example, if the server detects that the user is in poor health, it will automatically contact a medical institution by email or phone.

[2061] Prompt Sentence Examples

[2062] Prompt: "What questions should be generated when a user talks about a recent trip?"

[2063] Example output from a generative AI model: "Is there anywhere you'd like to travel to these days?"

[2064] This system enables the robot to understand the user's hobbies and health condition and take appropriate measures. The flow of the identification process in the third embodiment will be described with reference to FIG.

[2065] Step 1:

[2066] The device collects conversations with users as voice input. Specifically, when a user speaks to the robot, the robot captures the voice with a microphone and saves it as voice data.

[2067] Input: User speech

[2068] Output: Audio data

[2069] Step 2:

[2070] The device uses voice recognition software to convert the collected voice data into text data. For example, the voice recognition software analyzes the voice data and converts it into text such as "I want to go on a trip these days."

[2071] Input: Audio data

[2072] Output: Text data

[2073] Step 3:

[2074] The server then uses a natural language processing engine to extract keywords from the converted text data. Specifically, the natural language processing engine analyzes the text data and extracts keywords such as "travel" and "want to go."

[2075] Input: Text data

[2076] Output: Keywords

[2077] Step 4:

[2078] The server analyzes the extracted keywords and creates a user profile. For example, if the keyword "travel" appears frequently, the server records the user as "travel lover" in the user's profile.

[2079] Input: keyword

[2080] Output: User profile

[2081] Step 5:

[2082] The server uses a generative AI model to generate a conversation based on the user's profile. Specifically, the generative AI model generates a question based on the profile information, such as "Is there anywhere you'd like to travel to recently?"

[2083] Input: User Profile

[2084] Output: Generated conversation text

[2085] Step 6:

[2086] The device provides the generated conversation to the user as a voice output. Specifically, the device converts the generated text into voice using speech synthesis software and conveys the voice to the user.

[2087] Input: Generated conversation text

[2088] Output: Audio output

[2089] Step 7:

[2090] The device collects information about the user's health and lifestyle during conversations. For example, if the user says, "I haven't been feeling well lately," the device collects that information as voice data.

[2091] Input: User speech

[2092] Output: Audio data about health status

[2093] Step 8:

[2094] The server converts the collected voice data about health conditions into text data for analysis. Specifically, voice recognition software converts the voice data into text, and a natural language processing engine analyzes the text.

[2095] Input: Audio data about health conditions

[2096] Output: Text data about health conditions

[2097] Step 9:

[2098] The server will report the results of the health status analysis to medical institutions or local governments as necessary. Specifically, if the server detects an abnormality, it will automatically contact the medical institution by email or phone.

[2099] Input: Text data about health conditions

[2100] Output: Report to medical institutions and local authorities

[2101] (Application example 3)

[2102] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2103] In modern society, the number of single people and elderly people living alone is increasing, and their feelings of loneliness and how to respond in emergencies are becoming major issues. There is also a need to understand their hobbies and health conditions and provide appropriate support. However, conventional interactive robots have had difficulty fully learning about users' hobbies and health conditions and responding appropriately in emergencies.

[2104] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server is an interactive robot incorporating a generative AI, and is a system for selling to single people and elderly people living alone. The system includes: means for the robot to respond to everything from worries to casual everyday conversations, things to research, and purchasing desired products; means for enabling the robot's personality to be set in advance (age, conversational style); means for learning the user's hobbies, tastes, way of thinking, and outlook on life through repeated conversations; means for collecting information on the user's health condition and living situation and reporting it to medical institutions or local governments as necessary; and means for learning the user's hobbies and health condition through daily conversations and contacting medical institutions or local governments in case of an emergency. This makes it possible to understand the user's hobbies and health condition and provide appropriate support.

[2105] "Generative AI" is a system that uses artificial intelligence technology to generate new information and content from data.

[2106] An "interactive robot" is a robot that can have a natural conversation with a user.

[2107] "Single person" refers to someone who lives alone.

[2108] "Elderly people living alone" refers to elderly people who live alone.

[2109] A "system" is a mechanism in which multiple elements are combined to achieve a specific function.

[2110] "Problems" refer to problems or difficult situations that users face.

[2111] "Daily conversation" refers to general conversation in everyday life.

[2112] "What you want to find out" refers to the information or data that the user wants to know.

[2113] The "item you want to buy" refers to an item that you want to purchase.

[2114] "Personality setting" refers to setting a robot's characteristics, such as its age and how it speaks, in advance.

[2115] "Hobbies and tastes" refers to the activities and interests that users prefer.

[2116] "Mindset" refers to the user's thinking tendencies and values.

[2117] "Life perspective" refers to the user's perspective and feelings about life.

[2118] "Health Status" refers to the state of a user's physical and mental health.

[2119] "Living conditions" refers to the environment and circumstances of the user's daily life.

[2120] "Medical institution" refers to a facility that provides medical services, such as a hospital or clinic.

[2121] "Local government" refers to a local public body responsible for local administration.

[2122] "Emergency" means a situation...

Claims

1. A system incorporated in an interactive robot incorporating a generating AI, A means for enabling a personality of the interactive robot to be set in advance; A means to learn about the user's hobbies, tastes, way of thinking, and outlook on life through repeated conversations, means for determining an emotional state of a user using an emotion engine; a means for adjusting a conversation with a user so as to change the topic of conversation based on the learned hobbies, tastes, ways of thinking and outlook on life of the user when it is determined from the conversation that the user's emotional state is bored; A system including:

2. 2. The system of claim 1, further comprising means for notifying a medical institution when a sad emotion continues as the emotional state of the user for a certain period of time.

3. 2. The system according to claim 1, further comprising a means for reporting to a medical institution when a user's physical condition is detected from the contents of the conversation.

Images