System

Abstract

A system is provided.SOLUTION: The system includes a means for receiving an input from a user, a means for analyzing the user input and inquiring about a transmission destination and a message content, a generation artificial intelligence means for generating a message based on the transmission destination and the message content, a means for confirming the generated message to the user, and a means for transmitting the message after confirmation by the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Elderly people often have difficulty inputting text using digital devices such as smartphones and tablets, making sending messages to family and friends particularly cumbersome. Furthermore, when visual and auditory abilities decline, it becomes difficult to communicate smoothly using conventional input methods. Therefore, there is a need for a method that allows elderly people to easily send messages. [Means for solving the problem]

[0005] The present invention includes a means for accepting voice or text input from a user, analyzing it, and identifying the recipient and message content. It also includes a means for using generative artificial intelligence to generate an appropriate message based on the user's input, and prompts the user to confirm the generated message. After the user confirms the message, the system also includes a means for sending the message to the specified recipient. This allows even elderly people to easily send messages.

[0006] Specifically, the system includes a means for analyzing user input and a means for dynamically generating messages using generative artificial intelligence. Furthermore, the message sending means uses a communication service to actually send the message. The system supports both voice and text input, allowing the system to accurately generate and send the message content desired by the user.

[0007] "User" means a device user who uses the system to send messages.

[0008] "Input" refers to the act of a user providing information such as a destination or message content to the system, or that information.

[0009] "Analysis" is a series of processes that interprets input information and extracts necessary data.

[0010] "Destination" is information about the person to whom the user wants to send a message.

[0011] "Message content" refers to the information or text that the user wants to convey to the recipient.

[0012] "Generative AI" is an AI technology that has the ability to dynamically generate appropriate messages based on user input.

[0013] "Verification" is the process of having the user verify whether the generated message matches their intentions.

[0014] The "transmission means" refers to a function or method for actually transmitting the generated message to the designated destination.

[0015] "Communication services" means online platforms and infrastructure for sending and receiving messages over the internet and mobile networks.

[0016] A "system" is the totality of hardware and software combined to accept, analyze, and generate, validate, and transmit user input.

[0017] "Dynamically generated" refers to generating messages on the fly in response to user input, rather than using fixed text. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention relates to a system that enables elderly people to easily send messages using digital devices. Specific embodiments of the present invention will be described below.

[0040] 1. System Configuration

[0041] This system is broadly composed of the following components:

[0042] User devices: Devices such as smartphones and tablets used by seniors.

[0043] Server: The central control unit that handles the main processing of input analysis, message generation, and transmission.

[0044] Communication means: The Internet or mobile network to connect the server, user device, and destination.

[0045] 2. User Input

[0046] The user device has an interface for accepting voice or text input. The user starts a conversation with the system through the LINE app. At that time, the user uses voice or text input to indicate that they want to send a message to their family.

[0047] 3. Parsing the input by the server

[0048] The server analyzes the input sent by the user. This analysis uses voice recognition technology and natural language processing technology. For example, if the user instructs the server to "send a message to my son," the server recognizes that it needs to prompt the user to enter the recipient ("son") and the message content.

[0049] 4. Specify the recipient and message content

[0050] The server asks the user to choose a recipient and a message content. If the user selects "my son" as the recipient and enters "Let's go for a walk together next Sunday," the server creates a message based on this information.

[0051] 5. Message Generation Using Generative AI

[0052] The server uses generative artificial intelligence technology to generate an appropriate message based on the user's input. For example, a message such as "To your son: Let's go for a walk together this Sunday" is automatically generated.

[0053] 6. User Confirmation

[0054] The server presents the generated message to the user and asks for confirmation. If the user confirms and instructs transmission, the server executes the message transmission process.

[0055] 7. Server-sent messages

[0056] The server then uses an appropriate communication method to send the generated message to the specified destination, using the LINE API or similar to actually send the message.

[0057] 8. Examples:

[0058] Consider the case where a user opens the LINE app and says, "Tell my grandson happy birthday." The user's device receives this input as voice and sends the data to the server. The server converts the voice to text and identifies the recipient ("grandson") and the message content ("Happy Birthday"). The generative AI then generates the message "To Grandson: Happy Birthday" and asks the user for confirmation. Once the user has confirmed, the server actually sends the message via the LINE API.

[0059] Thus, the present invention provides a support system that enables elderly people to easily send messages, thereby enabling smooth communication.

[0060] The processing flow will be explained below.

[0061] Step 1:

[0062] The user launches the LINE app and starts a chat with the AI ​​concierge. The user requests to send a message by voice or text input.

[0063] Step 2:

[0064] The device accepts the user's voice or text input and sends it to the server. For example, the user may input "I want to send a message to my family."

[0065] Step 3:

[0066] The server analyzes the user's voice input and extracts the necessary data (recipient and message content) using voice recognition and natural language processing technologies.

[0067] Step 4:

[0068] The server generates a message asking for the recipient and sends it to the terminal. For example, the message "Who do you want to send a message to?" is displayed on the user's terminal.

[0069] Step 5:

[0070] The user specifies the recipient by voice or text, for example, typing "To my son."

[0071] Step 6:

[0072] The terminal sends the user's input to the server, which analyzes the input and checks the destination.

[0073] Step 7:

[0074] The server generates a message asking for the message content and sends it to the terminal. For example, the message "What message would you like to send to your son?" is displayed on the user terminal.

[0075] Step 8:

[0076] The user inputs the message content by voice or text, for example, "Let's go for a walk together this Sunday."

[0077] Step 9:

[0078] The terminal sends the user's input to the server, which analyzes the input and obtains the message content.

[0079] Step 10:

[0080] The server uses generative artificial intelligence to generate a message based on the recipient and message content, such as "To your son: Let's take a walk together this Sunday."

[0081] Step 11:

[0082] The server generates a message for the user to confirm the generated message and sends it to the terminal. For example, a confirmation message such as "Do you want to send the following message to your son? 'To your son: Let's take a walk together next Sunday'" is displayed on the user terminal.

[0083] Step 12:

[0084] The user checks the message content and instructs the sending of the message, for example, by typing "Yes, send it."

[0085] Step 13:

[0086] The terminal sends the user's confirmation input to the server, which receives the confirmation.

[0087] Step 14:

[0088] The server uses the LINE API to send a message to the specified recipient (son). The message is sent using the appropriate communication method.

[0089] Step 15:

[0090] The server checks whether the message was sent successfully.

[0091] Step 16:

[0092] The server generates a message notifying the user of the result of the transmission and sends it to the terminal. For example, a message saying "The message has been sent" is displayed on the user terminal.

[0093] The above steps make it possible for users to easily create and send messages.

[0094] Example 1

[0095] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0096] The present invention relates to a system for reducing the hassle and technical barriers that seniors face when sending messages using digital devices. Specifically, the purpose is to simplify the process by which seniors can easily create messages through voice input or text input and send them to specific recipients.

[0097] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0098] In this invention, the server includes input means for accepting input from the user, query means for analyzing the user input and querying the destination and message content, generative artificial intelligence means for generating a message based on the destination and message content, confirmation means for confirming the generated message with the user, and transmission means for transmitting the message after the user's confirmation. This provides an environment in which elderly users can easily send messages, enabling smoother communication.

[0099] "Input means for accepting input from the user" refers to an interface that allows a user to input information in voice or text format through a device such as a smartphone or tablet.

[0100] "Means for analyzing user input and inquiring about the destination and message content" refers to the process of analyzing received input data and asking the user questions to determine the destination and message content.

[0101] "Generative artificial intelligence means for generating messages based on the recipient and message content" refers to an artificial intelligence algorithm that utilizes natural language processing technology to generate an appropriate message based on user input.

[0102] The "means for confirming the generated message with the user" refers to a process of presenting the generated message to the user and receiving instructions for confirmation of the contents and corrections.

[0103] "Transmission means for transmitting a message after confirmation by the user" refers to a system function that actually transmits the message confirmed by the user to the designated destination.

[0104] "Communication Services" refers to the infrastructure for sending and receiving messages over the Internet and mobile networks.

[0105] "Voice input" refers to a method in which a user provides information by voice through a microphone.

[0106] "Text input" refers to a method in which a user provides textual information using a keyboard or touchscreen.

[0107] The present invention relates to a system that enables elderly people to easily send messages using digital devices. Specific embodiments of the present invention will be described below.

[0108] This system is broadly composed of the following components:

[0109] User devices: Devices such as smartphones and tablets used by seniors.

[0110] Server: The central control unit that handles the main processing of input analysis, message generation, and transmission.

[0111] Communication means: The Internet or mobile network to connect the server, user device, and destination.

[0112] The user terminal has an interface for accepting voice or text input. The user starts a dialogue with the system through a communication app. For example, the user may input "I want to send a message to my son" by voice.

[0113] The server first converts the voice data sent from the user device into text using speech recognition technology. For example, it uses a speech recognition engine such as Google® Cloud Speech-to-Text. Then, it uses natural language processing technology to analyze the user's intent, such as "I want to send a message to my son." Using a natural language processing engine such as Google Cloud Natural Language, it identifies the recipient and message content.

[0114] The server prompts the user to confirm the destination. For example, it might say, "Who would you like to send the message to?" If the user responds with "my son," the server then asks, "What would you like to say in the message?" The user might type, "Let's go for a walk together this Sunday." Based on this information, the server uses generative artificial intelligence technology (e.g., OpenAI's GPT-3) to generate an appropriate message.

[0115] The generated message will read, "To your son: Let's go for a walk together this Sunday." This message is presented to the user for confirmation. It is displayed with the prompt, "Are you sure you want to send this message?" If the user confirms "Yes," the server will actually send the message using a communication method (for example, LINE's API).

[0116] As a concrete example, consider the case where a user opens a communication app and says, "Tell my grandson happy birthday." The user device receives this input as voice and sends the data to the server. The server converts the voice to text using Google Cloud Speech-to-Text, analyzes it using Google Cloud Natural Language, and identifies the recipient ("grandson") and the message content ("happy birthday"). Then, a generative artificial intelligence (e.g., OpenAI GPT-3) generates the message "To Grandson: Happy Birthday" and asks the user for confirmation. Once the user has confirmed, the server sends the message via the LINE API.

[0117] Thus, the present invention provides a support system that enables elderly people to easily send messages, thereby enabling smooth communication.

[0118] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0119] Step 1:

[0120] The user inputs their intention to send a message in voice or text format on their smartphone, tablet, or other device (user device). For example, they might say, "I want to send a message to my son." The device captures this input and sends it to the server as digital data.

[0121] Input: User voice or text input

[0122] Output: Input contents in digital data format

[0123] Step 2:

[0124] The server receives the voice data sent from the device and converts it into text using speech recognition technology. Specifically, it uses a speech recognition engine such as Google Cloud Speech-to-Text. It then analyzes the input using natural language processing technology. Using Google Cloud Natural Language or similar, it identifies the recipient ("son") and the message content ("I want to send a message").

[0125] Input: Audio input in digital data format

[0126] Output: Analysis results in text format

[0127] Step 3:

[0128] Based on the analysis results, the server displays a prompt on the terminal to the user to confirm the destination. It notifies the user, "Please tell us who you want to send the message to." If the user responds with "my son," it then displays a prompt, "Please tell us the content of the message you want to send."

[0129] Input: Text data processed by speech recognition and natural language processing

[0130] Output: Prompts for destination and message content

[0131] Step 4:

[0132] When a user types "Let's go for a walk together this Sunday," the server uses generative AI to prepare a prompt to generate an appropriate message, specifically using a generative AI model such as OpenAI GPT-3.

[0133] Input: The recipient and message content entered by the user

[0134] Output: Prompt for the generative AI

[0135] Step 5:

[0136] The server sends a prompt to the generative AI and receives a generated message, such as "To your son: Let's take a walk together this Sunday."

[0137] Input: prompt and generative AI model

[0138] Output: The generated message

[0139] Step 6:

[0140] The server sends the generated message to the user's terminal, asking for user confirmation. The message is displayed with the prompt "Are you sure you want to send this message?" The user confirms and replies "Yes."

[0141] Input: The generated message

[0142] Output: Confirmation prompt and confirmation result

[0143] Step 7:

[0144] The server sends the message to the specified destination using an appropriate communication method such as the LINE API. If the message is sent successfully, a confirmation message stating "The message has been sent" is displayed to the user.

[0145] Input: User confirmation result

[0146] Output: Actual message sent and confirmation message

[0147] Through the above steps, this system helps elderly people to send messages easily.

[0148] (Application example 1)

[0149] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0150] When elderly people use digital devices to set destinations or send messages, many of the operations are cumbersome and difficult to use. This problem is particularly pronounced when using autonomous vehicles, making it difficult for elderly people to use them with confidence. Furthermore, there are limited input methods for accurately conveying intended content, so accurate message generation and transmission cannot be guaranteed.

[0151] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0152] In this invention, the server includes means for accepting input from a user, means for analyzing the user input and inquiring about the destination and message content, generative artificial intelligence means for generating a message based on the destination and message content, means for confirming the generated message with the user, means for sending the message after the user's confirmation, means for the user terminal to accept voice input, and means for setting a destination by voice input. This enables elderly people to easily set a destination and send a message by voice input in an autonomous vehicle.

[0153] A "means for accepting input from a user" is an interface for receiving user input in the form of voice or text.

[0154] The "means for analyzing user input and inquiring about the destination and message content" is a processing means for analyzing the input from the user and identifying the necessary information (destination and message content).

[0155] The "generative artificial intelligence means for generating a message based on the destination and message content" is an artificial intelligence technology for generating an appropriate message based on the destination and message content designated by the user.

[0156] The "means for confirming the generated message with the user" is a means for presenting the contents of the generated message to the user and requesting confirmation.

[0157] The "means for sending a message after user confirmation" is a means for sending a message that has been confirmed by the user to a specified destination.

[0158] The "means by which the user terminal accepts voice input" refers to the means by which the user terminal recognizes voice and receives the content of the voice.

[0159] The "means for setting a destination by voice input" is a means for analyzing a voice input and setting a destination for the vehicle.

[0160] The system for carrying out the present invention accepts input from a user, creates a message based on the input, sets a destination, and sends the message. A method for realizing this system will be described in detail below.

[0161] System Configuration

[0162] This system is broadly composed of the following components:

[0163] 1. User Device

[0164] 2. Server

[0165] 3. Means of communication

[0166] User terminal

[0167] The user terminal is a smartphone or tablet used by the elderly, equipped with a microphone for accepting voice input, and with software for performing voice recognition (e.g., a speech_recognition library) installed.

[0168] server

[0169] The server analyzes the user's input and identifies the recipient and message content. Specifically, it uses voice recognition technology to convert the user's voice input into text data, and then uses natural language processing technology to extract the recipient and message content.

[0170] The server has the following main functions:

[0171] 1. Speech recognition technology: Use the speech_recognition library to convert voice data into text data.

[0172] 2. Natural language processing technology: Generative AI models are used to generate appropriate messages from user voice input, such as instructions for setting a destination or extracting message content.

[0173] 3. Message generation: Create a message based on the generated text data and ask the user for confirmation.

[0174] 4. Send message: Once confirmed, the message is sent using a communication method such as the LINE API.

[0175] communication means

[0176] The Internet and mobile networks are used as communication means, enabling data communication between the server and user terminals, and between message destinations.

[0177] Specific examples

[0178] Prompt Sentence Examples

[0179] Consider the case where a user speaks the following:

[0180] "Please let my daughter know her doctor's appointment time next week."

[0181] 1. Voice input

[0182] The user inputs voice commands in the car such as, "Please tell my daughter when she will go to the hospital next week." The user terminal receives this voice command and collects the voice data through the microphone.

[0183] 2. Data Analysis and Transformation

[0184] The user device sends the collected voice data to the server, which uses the speech_recognition library to convert the voice data into text data and then uses natural language processing technology to extract the recipient ("daughter") and the message content ("Please let me know her doctor's appointment time next week").

[0185] 3. Message Creation

[0186] Based on the generative artificial intelligence model, the server generates an appropriate message such as "Dear daughter: Please let me know when she will be coming to the hospital next week." The generated message is then sent to the user's device, where the user is asked to confirm it.

[0187] 4. Sending a message

[0188] Once the user has confirmed the details, the server will send the message to the daughter using the LINE API, which allows for smooth destination setting and message sending.

[0189] In this way, the system of the present invention is designed to enable elderly people to easily use advanced technology, allowing users to use self-driving vehicles with peace of mind and send necessary messages accurately.

[0190] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0191] Step 1:

[0192] The user inputs voice and the device receives the voice. The voice input uses a prompt such as "Please tell my daughter when she will be going to the doctor next week." The device uses a microphone to collect the voice data and temporarily stores it.

[0193] Input: Voice input from the user

[0194] Output: Audio data

[0195] Step 2:

[0196] The device sends the collected voice data to a server, where it uploads the voice data to the server via the Internet or a mobile network.

[0197] Input: Audio data

[0198] Output: Audio data sent to the server

[0199] Step 3:

[0200] The server analyzes the received voice data and converts it into text data. Specifically, it uses the speech_recognition library to convert the voice into text. This process results in the text data "Please let my daughter know when she will go to the hospital next week."

[0201] Input: Audio data

[0202] Output: Text data

[0203] Step 4:

[0204] The server analyzes the text data and extracts the recipient and message content. Using a generative AI model, it identifies "daughter" as the recipient and "let me know when she'll be going to the hospital next week" as the message content.

[0205] Input: Text data

[0206] Output: destination and message content

[0207] Step 5:

[0208] The server generates an appropriate message based on the recipient and message content, for example, "To your daughter: Please let me know when she will be coming to the hospital next week."

[0209] Input: Recipient and message content

[0210] Output: The generated message

[0211] Step 6:

[0212] The server sends the generated message to the user terminal for confirmation, and the terminal displays the message on its screen for the user to confirm.

[0213] Input: The generated message

[0214] Output: Message sent to the user's terminal

[0215] Step 7:

[0216] The user confirms the displayed message and instructs the sending. When the user presses the confirmation button, the terminal sends the instruction to the server.

[0217] Input: User confirmation prompt

[0218] Output: Confirmation instructions sent to the server

[0219] Step 8:

[0220] After the server confirms the user's identity, it sends the message to the specified destination using the LINE API, etc. The communication method is the internet or mobile network.

[0221] Input: Acknowledged message

[0222] Output: Message sent to destination

[0223] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0224] The present invention relates to a system that allows elderly people to easily send messages using digital devices. In particular, the system incorporates an emotion engine that recognizes the user's emotions and can adjust the content of messages according to the user's emotions.

[0225] 1. System Configuration

[0226] This system consists of the following main components:

[0227] User devices: Devices such as smartphones and tablets used by seniors.

[0228] Server: The central control unit that handles the main processing of input analysis, emotion recognition, message generation, and transmission.

[0229] Communication means: The Internet or mobile network to connect the server, user device, and destination.

[0230] Emotion engine: Software capable of recognizing emotions from user input and adjusting processing accordingly.

[0231] 2. User Input

[0232] The user device has an interface for accepting voice or text input. The user starts a conversation with the system through the LINE app. At that time, the user uses voice or text input to indicate that they want to send a message to their family.

[0233] 3. Parsing the input by the server

[0234] The server analyzes the input sent by the user. This analysis uses voice recognition technology and natural language processing technology. For example, if the user instructs the server to "send a message to my son," the server recognizes that it needs to prompt the user to enter the recipient ("son") and the message content.

[0235] 4. Emotion Recognition by Emotion Engine

[0236] The server uses an emotion engine to recognize emotions from the voice and text input by the user. The emotion engine analyzes the emotional characteristics of the user's speech and text and identifies emotions such as joy, sadness, and anger.

[0237] 5. Specify recipient and message content

[0238] The server asks the user to choose a recipient and a message content. If the user selects "my son" as the recipient and enters "Let's go for a walk together next Sunday," the server creates a message based on this information.

[0239] 6. Message Generation Using Generative AI

[0240] The server uses generative artificial intelligence technology to generate an appropriate message based on the user's input and the emotion recognition results. The message content is adjusted according to the analysis results of the emotion engine. For example, if the user is feeling happy, the server generates a message such as "To your son: I'm so happy! Let's take a walk together next Sunday."

[0241] 7. User Confirmation

[0242] The server presents the generated message to the user and asks for confirmation. If the user confirms and instructs transmission, the server executes the message transmission process.

[0243] 8. Server-sent messages

[0244] The server then uses an appropriate communication method to send the generated message to the specified destination, using the LINE API or similar to actually send the message.

[0245] 9. Examples:

[0246] Consider the case where a user opens the LINE app and says, "Tell my grandson happy birthday." The user's device receives this input as voice and sends the data to the server. The server converts the voice into text, analyzes it, and identifies the recipient ("grandson") and the message content ("Happy Birthday"). Next, the emotion engine recognizes the user's emotion of joy. After that, the generative AI generates a message saying, "To my grandson: Happy birthday! I'm so happy," and asks the user for confirmation. Once the user has confirmed, the server actually sends the message via the LINE API.

[0247] In this way, the present invention provides a support system that enables elderly people to easily send messages, and by combining it with an emotion engine, it becomes possible to send more personalized messages.

[0248] The processing flow will be explained below.

[0249] Step 1:

[0250] The user launches the LINE app and starts chatting with the AI ​​concierge. The user requests to send a message by voice or text input. For example, the user might type, "I want to send a message to my family."

[0251] Step 2:

[0252] The terminal accepts the user's voice or text input and sends it to the server.

[0253] Step 3:

[0254] The server analyzes the user's voice input and extracts the necessary data (recipient and message content). Speech recognition technology and natural language processing technology are used for the analysis. For example, it might extract "I want to send a message to my family."

[0255] Step 4:

[0256] The server runs an emotion engine to recognize emotions from the user's voice and text input, for example, recognizing "joy" from the user's tone of voice and the content of the text.

[0257] Step 5:

[0258] The server generates a message asking for the recipient and sends it to the terminal. For example, the message "Who do you want to send a message to?" is displayed on the user's terminal.

[0259] Step 6:

[0260] The user specifies the recipient by voice or text, for example, typing "To my son."

[0261] Step 7:

[0262] The terminal sends the user's input to the server, which analyzes the input and identifies the destination.

[0263] Step 8:

[0264] The server generates a message asking for the message content and sends it to the terminal. For example, the message "What message would you like to send to your son?" is displayed on the user terminal.

[0265] Step 9:

[0266] The user inputs the message content by voice or text, for example, "Let's go for a walk together this Sunday."

[0267] Step 10:

[0268] The terminal sends the user's input to the server, which analyzes the input and obtains the message content.

[0269] Step 11:

[0270] The server uses generative artificial intelligence to generate messages based on the recipient and message content. It also adjusts the message content based on the analysis results of the emotion engine. For example, it might generate a message like, "Dear son: How are you? I'm so happy! Let's take a walk together this Sunday."

[0271] Step 12:

[0272] The server generates a message for the user to confirm the generated message and sends it to the terminal. For example, a confirmation message such as "Would you like to send the following message to your son? 'To your son: How are you? I'm so happy! Let's go for a walk together next Sunday'" is displayed on the user terminal.

[0273] Step 13:

[0274] The user checks the message content and instructs the sending of the message, for example, by typing "Yes, send it."

[0275] Step 14:

[0276] The terminal sends the user's confirmation input to the server, which receives the confirmation.

[0277] Step 15:

[0278] The server uses the LINE API to send a message to the specified recipient (son). The message is sent using the appropriate communication method.

[0279] Step 16:

[0280] The server checks whether the message was sent successfully.

[0281] Step 17:

[0282] The server generates a message notifying the user of the result of the transmission and sends it to the terminal. For example, a message saying "The message has been sent" is displayed on the user terminal.

[0283] The above steps make it possible for users to easily create messages, adjust them according to their emotions, and send them.

[0284] Example 2

[0285] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0286] For seniors, the process of sending messages using digital devices can be complicated, and sending comments that don't reflect their emotions can be stressful. Furthermore, conventional messaging systems lack the ability to analyze users' emotions and tailor messages accordingly. Therefore, the present invention aims to provide a system that allows seniors to send messages more easily and that appropriately reflect their emotions.

[0287] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0288] In this invention, the server includes means for accepting input from a user, means for analyzing the user input and inquiring about the recipient and message content, means for recognizing emotions, generative artificial intelligence means for generating a message based on the recipient, message content, and emotion recognition results, means for confirming the generated message with the user, and means for sending the message after the user's confirmation. This makes it possible for elderly people to easily use digital devices and send messages that reflect their emotions.

[0289] "Means for accepting input from the user" refers to the function of receiving data on a digital device when a user inputs voice or text.

[0290] The "means for analyzing user input and inquiring about the recipient and message content" is a function that analyzes the information entered by the user and asks the user to confirm the recipient and content of the message.

[0291] "Means for recognizing emotions" refers to a function for identifying emotions from a user's voice or text data, and is a technology for analyzing the user's emotional state.

[0292] "Generative AI means" refers to an AI technology that automatically creates appropriate and personalized messages based on user input and emotion recognition results.

[0293] The "means for having the user confirm the generated message" is a function for displaying the generated message content to the user, requesting confirmation, and accepting instructions for correction or transmission.

[0294] The "means for sending a message after confirmation by the user" is a function for sending a message confirmed by the user to a specified destination using an appropriate communication means.

[0295] The embodiments of the present invention will be specifically described below.

[0296] This system is designed to enable elderly people to easily send messages using digital devices. The main components of the system are a user terminal, a server, a communication method, and an emotion engine.

[0297] The user devices are devices such as smartphones and tablets used by the elderly, and are equipped with an interface that accepts voice and text input. Users can access the system through the LINE app and send messages by voice input.

[0298] The server serves as the central control unit for the system, handling key processes such as input analysis, emotion recognition, message generation, and transmission. It uses Google Cloud Speech-to-Text API for speech recognition and Microsoft® Azure® Text Analytics emotion analysis for emotion recognition. It also uses generative artificial intelligence such as OpenAI GPT-4® for message generation.

[0299] Specifically, when a user opens the LINE app and says, "Tell my grandson happy birthday," the user's device accepts this voice input. The voice data is sent over the Internet to a server, which then converts it into text using the Google Cloud Speech-to-Text API.

[0300] The server then inputs the text data into Microsoft Azure Text Analytics Sentiment Analysis to recognize the user's emotion (in this case, joy). The server then uses OpenAI GPT-4 to generate an appropriate message based on the emotion recognition results. For example, the message generated might be, "Dear Grandson: Happy Birthday! I'm so happy."

[0301] The generated message is sent via the LINE app, and if the user confirms and instructs it to be sent, the server actually sends the message using the LINE API.

[0302] Examples of prompt sentences include the following:

[0303] "Please send a birthday message to my grandson."

[0304] "Tell my son I want him to come over."

[0305] In this way, the system allows seniors to easily send personalized messages that appropriately reflect their emotions.

[0306] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0307] Step 1:

[0308] Users launch the LINE app on their smartphone or tablet and use voice or text input.

[0309] Specifically, the user presses the voice input button on the LINE app and says, "Tell my grandson happy birthday."

[0310] As input, the user's voice data is taken into the terminal.

[0311] Step 2:

[0312] The device temporarily stores the voice data and transmits it to a server via the Internet.

[0313] Specifically, the terminal sends voice data to the specified URL of the server. The voice data is sent to the server as input.

[0314] Step 3:

[0315] The server converts the received voice data into text data using the Google Cloud Speech-to-Text API.

[0316] As input, speech data is given, and as output, text data such as "Tell my grandson happy birthday" is obtained.

[0317] The specific operation of the server is to call the Google Cloud Speech-to-Text API and convert the voice data into text.

[0318] Step 4:

[0319] The server analyzes the text data and extracts the information necessary to verify the recipient and message content.

[0320] As input, text data "Tell my grandchild happy birthday" is given, and as output, the recipient "grandchild" and the message content "Happy Birthday" are extracted.

[0321] Specifically, the server uses natural language processing technology to analyze the text and extract the required information.

[0322] Step 5:

[0323] The server inputs the text data into Microsoft Azure Text Analytics sentiment analysis to recognize the user's emotions.

[0324] The input is the text data "Tell my grandson happy birthday," and the output is the emotion "joy."

[0325] Specifically, the server calls the emotion engine and recognizes emotions from text data.

[0326] Step 6:

[0327] The server generates an appropriate message using generative artificial intelligence (OpenAI GPT-4) based on the recipient, message content, and emotion recognition results.

[0328] The inputs are given as the recipient "grandchild", the message content "Happy Birthday", and the emotion "Joy", and the output is generated as the message "To Grandchild: Happy Birthday! I'm so happy."

[0329] Specifically, the server calls the generative artificial intelligence and generates an appropriate message.

[0330] Step 7:

[0331] The server returns the generated message to the user's LINE app and presents it to them for confirmation.

[0332] The generated message "To my grandson: Happy birthday! I'm so happy" is given as input, and confirmation is requested from the user as output.

[0333] Specifically, the server uses the LINE API to send a message to the user's LINE app and displays a confirmation button.

[0334] Step 8:

[0335] The user checks the message on the LINE app and instructs it to be sent.

[0336] The message generated as input is displayed and the user presses the send confirmation button.

[0337] As a specific operation, the user presses the transmission confirmation button to issue a transmission instruction.

[0338] Step 9:

[0339] After the user confirms, the server uses the LINE API to send the message to the specified destination.

[0340] As input, the destination "Grandchild" and the generated message "To Grandchild: Happy Birthday! I'm so happy" are given, and as output the message is actually sent.

[0341] Specifically, the server calls the LINE API and sends the message to the destination.

[0342] (Application example 2)

[0343] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0344] When elderly people use food delivery services, they face the problem of difficulty in ordering due to the complexity of the operation. In addition, they face the problem of low user satisfaction due to the inability to provide personalized orders that take into account the user's emotions. A system that can solve these problems is needed.

[0345] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0346] In this invention, the server includes means for accepting input from a user, means for analyzing the user input and inquiring about the recipient and message content, generative artificial intelligence means for generating a message based on the recipient and message content, means for confirming the generated message with the user, means for sending the message after the user confirms, emotion recognition means for analyzing the emotion of the user input, and means for adjusting the message content based on the emotion recognition result. This not only enables elderly people to easily order food delivery, but also enables personalized order content based on their emotions.

[0347] The "means for accepting input from a user" is an interface for obtaining input data from a user in the form of voice or text.

[0348] The "means for analyzing user input and inquiring about the destination and message content" is a processing device for analyzing user input, extracting necessary information, and confirming the destination and message content.

[0349] "Generative artificial intelligence means for generating messages based on the recipient and message content" refers to artificial intelligence technology for automatically generating appropriate messages based on user input data and emotion recognition results.

[0350] The "means for confirming the generated message to the user" is an interface for displaying the generated message to the user and requesting confirmation of the contents.

[0351] The "means for sending a message after confirmation by the user" is a communication means for sending a message confirmed by the user to a specified destination.

[0352] The "emotion recognition means for analyzing the emotion of the user input" is an analysis engine for identifying the emotion from the voice or text input by the user.

[0353] The "means for adjusting the message content based on the emotion recognition result" is a processing device for appropriately changing the content and expression of the message based on the recognized emotion.

[0354] This invention is a system that allows elderly people to easily order food delivery, and can analyze the user's emotions to provide personalized ordering. The system consists of the following main components:

[0355] 1. A means of accepting input from the user

[0356] The user terminal accepts input from the user either by voice input or text input. This is typically a smartphone or tablet, and these devices have built-in microphones.

[0357] 2. A way to parse user input and query the recipient and message content

[0358] The server first analyzes the voice or text sent by the user. In the case of voice input, speech recognition technology (for example, the speech_recognition library) is used to convert the voice to text. Then, natural language processing technology is used to identify the recipient and message content.

[0359] 3. A method for adjusting message content based on emotion recognition results

[0360] The server analyzes the data entered by the user and identifies the user's emotion using an emotion recognition engine (e.g., emotion_recognition library). Based on the recognized emotion, the server adjusts the message content.

[0361] 4. Generative AI means for generating messages

[0362] The server generates appropriate messages based on user input and emotion recognition results, using a generative AI model, and the generated messages are personalized based on the user's emotional state.

[0363] 5. A means of confirming generated messages to the user

[0364] The server presents the generated message to the user and asks for confirmation, and if the user confirms and approves the sending, the message is actually sent.

[0365] 6. Means of sending messages

[0366] The server then sends the confirmed message to the specified destination using a communication method (e.g., the Internet or a mobile network) using a food delivery API.

[0367] For example, a user might say, "Tempura udon, please." The server converts this speech into text and analyzes it. The emotion recognition engine then identifies the emotion "hurry." Based on this information, the server responds to the user, saying, "It seems you're in a hurry. Here are some recommended express menu items."

[0368] Example prompts for generative AI models

[0369] For example, if a user says, "Tempura udon, please," we convert the speech input to text and use the emotion engine to identify the emotion "hurry." We then respond to the user with the following: "Sounds like you're in a hurry. Here are some recommended express menu items:..."

[0370] In this way, the present invention makes it easier and more comfortable for seniors to use food delivery services and personalizes the experience.

[0371] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0372] Step 1:

[0373] The user starts the food delivery application and inputs the phrase "Tempura udon please" by voice or text. The device acquires this input data.

[0374] Input: User voice or text input

[0375] Output: Audio or text data

[0376] Step 2:

[0377] The device sends the acquired voice data to the server, which converts the voice data into text using voice recognition technology (speech_recognition library).

[0378] Input: Audio data

[0379] Output: Text data

[0380] Step 3:

[0381] The server analyzes the text data using natural language processing technology to identify the recipient and message content. The analysis results in the recipient (e.g., food delivery service) and message content (e.g., ordering tempura udon).

[0382] Input: Text data

[0383] Output: Recipient information and message content

[0384] Step 4:

[0385] The server uses an emotion recognition engine (emotion_recognition library) to analyze emotions from the user's text data. In this step, the user's emotional state, such as whether they are in a hurry or happy, is identified.

[0386] Input: Text data

[0387] Output: Emotional state

[0388] Step 5:

[0389] The server adjusts the message content based on the user's emotional state. For example, if the user is in a hurry, it generates a response such as, "It seems you're in a hurry. Here are some recommended express delivery options." Generative AI technology is used to create messages tailored to the user.

[0390] Input: Message content and emotional state

[0391] Output: Adjusted message content

[0392] Step 6:

[0393] The server presents the generated message to the user and asks for confirmation of the contents. The terminal displays this message to the user and accepts confirmation input.

[0394] Input: Adjusted message content

[0395] Output: User confirmation input

[0396] Step 7:

[0397] The user checks the message content and approves the sending. The terminal sends this confirmation data to the server.

[0398] Input: User confirmation input

[0399] Output: Confirmation data

[0400] Step 8:

[0401] The server sends the confirmed message to the specified destination, and executes the order via the food delivery API using the communication method (Internet or mobile network).

[0402] Input: Confirmation data and message content

[0403] Output: Food delivery order confirmation

[0404] This allows users, even seniors, to easily order food delivery and personalize the experience based on their emotions.

[0405] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0406] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0407] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0408] [Second embodiment]

[0409] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0410] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0411] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0412] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0413] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0414] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0415] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0416] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0417] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0418] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0419] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0420] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0421] The present invention relates to a system that enables elderly people to easily send messages using digital devices. Specific embodiments of the present invention will be described below.

[0422] 1. System Configuration

[0423] This system is broadly composed of the following components:

[0424] User devices: Devices such as smartphones and tablets used by seniors.

[0425] Server: The central control unit that handles the main processing of input analysis, message generation, and transmission.

[0426] Communication means: The Internet or mobile network to connect the server, user device, and destination.

[0427] 2. User Input

[0428] The user device has an interface for accepting voice or text input. The user starts a conversation with the system through the LINE app. At that time, the user uses voice or text input to indicate that they want to send a message to their family.

[0429] 3. Parsing the input by the server

[0430] The server analyzes the input sent by the user. This analysis uses voice recognition technology and natural language processing technology. For example, if the user instructs the server to "send a message to my son," the server recognizes that it needs to prompt the user to enter the recipient ("son") and the message content.

[0431] 4. Specify the recipient and message content

[0432] The server asks the user to choose a recipient and a message content. If the user selects "my son" as the recipient and enters "Let's go for a walk together next Sunday," the server creates a message based on this information.

[0433] 5. Message Generation Using Generative AI

[0434] The server uses generative artificial intelligence technology to generate an appropriate message based on the user's input. For example, a message such as "To your son: Let's go for a walk together this Sunday" is automatically generated.

[0435] 6. User Confirmation

[0436] The server presents the generated message to the user and asks for confirmation. If the user confirms and instructs transmission, the server executes the message transmission process.

[0437] 7. Server-sent messages

[0438] The server then uses an appropriate communication method to send the generated message to the specified destination, using the LINE API or similar to actually send the message.

[0439] 8. Examples:

[0440] Consider the case where a user opens the LINE app and says, "Tell my grandson happy birthday." The user's device receives this input as voice and sends the data to the server. The server converts the voice to text and identifies the recipient ("grandson") and the message content ("Happy Birthday"). The generative AI then generates the message "To Grandson: Happy Birthday" and asks the user for confirmation. Once the user has confirmed, the server actually sends the message via the LINE API.

[0441] Thus, the present invention provides a support system that enables elderly people to easily send messages, thereby enabling smooth communication.

[0442] The processing flow will be explained below.

[0443] Step 1:

[0444] The user launches the LINE app and starts a chat with the AI ​​concierge. The user requests to send a message by voice or text input.

[0445] Step 2:

[0446] The device accepts the user's voice or text input and sends it to the server. For example, the user may input "I want to send a message to my family."

[0447] Step 3:

[0448] The server analyzes the user's voice input and extracts the necessary data (recipient and message content) using voice recognition and natural language processing technologies.

[0449] Step 4:

[0450] The server generates a message asking for the recipient and sends it to the terminal. For example, the message "Who do you want to send a message to?" is displayed on the user's terminal.

[0451] Step 5:

[0452] The user specifies the recipient by voice or text, for example, typing "To my son."

[0453] Step 6:

[0454] The terminal sends the user's input to the server, which analyzes the input and checks the destination.

[0455] Step 7:

[0456] The server generates a message asking for the message content and sends it to the terminal. For example, the message "What message would you like to send to your son?" is displayed on the user terminal.

[0457] Step 8:

[0458] The user inputs the message content by voice or text, for example, "Let's go for a walk together this Sunday."

[0459] Step 9:

[0460] The terminal sends the user's input to the server, which analyzes the input and obtains the message content.

[0461] Step 10:

[0462] The server uses generative artificial intelligence to generate a message based on the recipient and message content, such as "To your son: Let's take a walk together this Sunday."

[0463] Step 11:

[0464] The server generates a message for the user to confirm the generated message and sends it to the terminal. For example, a confirmation message such as "Do you want to send the following message to your son? 'To your son: Let's take a walk together next Sunday'" is displayed on the user terminal.

[0465] Step 12:

[0466] The user checks the message content and instructs the sending of the message, for example, by typing "Yes, send it."

[0467] Step 13:

[0468] The terminal sends the user's confirmation input to the server, which receives the confirmation.

[0469] Step 14:

[0470] The server uses the LINE API to send a message to the specified recipient (son). The message is sent using the appropriate communication method.

[0471] Step 15:

[0472] The server checks whether the message was sent successfully.

[0473] Step 16:

[0474] The server generates a message notifying the user of the result of the transmission and sends it to the terminal. For example, a message saying "The message has been sent" is displayed on the user terminal.

[0475] The above steps make it possible for users to easily create and send messages.

[0476] Example 1

[0477] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0478] The present invention relates to a system for reducing the hassle and technical barriers that seniors face when sending messages using digital devices. Specifically, the purpose is to simplify the process by which seniors can easily create messages through voice input or text input and send them to specific recipients.

[0479] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0480] In this invention, the server includes input means for accepting input from the user, query means for analyzing the user input and querying the destination and message content, generative artificial intelligence means for generating a message based on the destination and message content, confirmation means for confirming the generated message with the user, and transmission means for transmitting the message after the user's confirmation. This provides an environment in which elderly users can easily send messages, enabling smoother communication.

[0481] "Input means for accepting input from the user" refers to an interface that allows a user to input information in voice or text format through a device such as a smartphone or tablet.

[0482] "Means for analyzing user input and inquiring about the destination and message content" refers to the process of analyzing received input data and asking the user questions to determine the destination and message content.

[0483] "Generative artificial intelligence means for generating messages based on the recipient and message content" refers to an artificial intelligence algorithm that utilizes natural language processing technology to generate an appropriate message based on user input.

[0484] The "means for confirming the generated message with the user" refers to a process of presenting the generated message to the user and receiving instructions for confirmation of the contents and corrections.

[0485] "Transmission means for transmitting a message after confirmation by the user" refers to a system function that actually transmits the message confirmed by the user to the designated destination.

[0486] "Communication Services" refers to the infrastructure for sending and receiving messages over the Internet and mobile networks.

[0487] "Voice input" refers to a method in which a user provides information by voice through a microphone.

[0488] "Text input" refers to a method in which a user provides textual information using a keyboard or touchscreen.

[0489] The present invention relates to a system that enables elderly people to easily send messages using digital devices. Specific embodiments of the present invention will be described below.

[0490] This system is broadly composed of the following components:

[0491] User devices: Devices such as smartphones and tablets used by seniors.

[0492] Server: The central control unit that handles the main processing of input analysis, message generation, and transmission.

[0493] Communication means: The Internet or mobile network to connect the server, user device, and destination.

[0494] The user terminal has an interface for accepting voice or text input. The user starts a dialogue with the system through a communication app. For example, the user may input "I want to send a message to my son" by voice.

[0495] The server first converts the voice data sent from the user's device into text using speech recognition technology. For example, it uses a speech recognition engine such as Google Cloud Speech-to-Text. It then uses natural language processing technology to analyze the user's intent, such as "I want to send a message to my son." Using a natural language processing engine such as Google Cloud Natural Language, it identifies the recipient and message content.

[0496] The server prompts the user to confirm the destination. For example, it might say, "Who would you like to send the message to?" If the user responds with "my son," the server then asks, "What would you like to say in the message?" The user might type, "Let's go for a walk together this Sunday." Based on this information, the server uses generative artificial intelligence techniques (e.g., OpenAI's GPT-3) to generate an appropriate message.

[0497] The generated message will read, "To your son: Let's go for a walk together this Sunday." This message is presented to the user for confirmation. It is displayed with the prompt, "Are you sure you want to send this message?" If the user confirms "Yes," the server will actually send the message using a communication method (for example, LINE's API).

[0498] As a concrete example, consider the case where a user opens a communication app and says, "Tell my grandson happy birthday." The user device receives this input as voice and sends the data to the server. The server converts the voice to text using Google Cloud Speech-to-Text, analyzes it using Google Cloud Natural Language, and identifies the recipient ("grandson") and the message content ("happy birthday"). Then, a generative artificial intelligence (e.g., OpenAI GPT-3) generates the message "To Grandson: Happy Birthday" and asks the user for confirmation. Once the user has confirmed, the server sends the message via the LINE API.

[0499] Thus, the present invention provides a support system that enables elderly people to easily send messages, thereby enabling smooth communication.

[0500] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0501] Step 1:

[0502] The user inputs their intention to send a message in voice or text format on their smartphone, tablet, or other device (user device). For example, they might say, "I want to send a message to my son." The device captures this input and sends it to the server as digital data.

[0503] Input: User voice or text input

[0504] Output: Input contents in digital data format

[0505] Step 2:

[0506] The server receives the voice data sent from the device and converts it into text using speech recognition technology. Specifically, it uses a speech recognition engine such as Google Cloud Speech-to-Text. It then analyzes the input using natural language processing technology. Using Google Cloud Natural Language or similar, it identifies the recipient ("son") and the message content ("I want to send a message").

[0507] Input: Audio input in digital data format

[0508] Output: Analysis results in text format

[0509] Step 3:

[0510] Based on the analysis results, the server displays a prompt on the terminal to the user to confirm the destination. It notifies the user, "Please tell us who you want to send the message to." If the user responds with "my son," it then displays a prompt, "Please tell us the content of the message you want to send."

[0511] Input: Text data processed by speech recognition and natural language processing

[0512] Output: Prompts for destination and message content

[0513] Step 4:

[0514] When a user types "Let's go for a walk together this Sunday," the server uses generative AI to prepare a prompt to generate an appropriate message, specifically using a generative AI model such as OpenAI GPT-3.

[0515] Input: The recipient and message content entered by the user

[0516] Output: Prompt for the generative AI

[0517] Step 5:

[0518] The server sends a prompt to the generative AI and receives a generated message, such as "To your son: Let's take a walk together this Sunday."

[0519] Input: prompt and generative AI model

[0520] Output: The generated message

[0521] Step 6:

[0522] The server sends the generated message to the user's terminal, asking for user confirmation. The message is displayed with the prompt "Are you sure you want to send this message?" The user confirms and replies "Yes."

[0523] Input: The generated message

[0524] Output: Confirmation prompt and confirmation result

[0525] Step 7:

[0526] The server sends the message to the specified destination using an appropriate communication method such as the LINE API. If the message is sent successfully, a confirmation message stating "The message has been sent" is displayed to the user.

[0527] Input: User confirmation result

[0528] Output: Actual message sent and confirmation message

[0529] Through the above steps, this system helps elderly people to send messages easily.

[0530] (Application example 1)

[0531] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0532] When elderly people use digital devices to set destinations or send messages, many of the operations are cumbersome and difficult to use. This problem is particularly pronounced when using autonomous vehicles, making it difficult for elderly people to use them with confidence. Furthermore, there are limited input methods for accurately conveying intended content, so accurate message generation and transmission cannot be guaranteed.

[0533] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0534] In this invention, the server includes means for accepting input from a user, means for analyzing the user input and inquiring about the destination and message content, generative artificial intelligence means for generating a message based on the destination and message content, means for confirming the generated message with the user, means for sending the message after the user's confirmation, means for the user terminal to accept voice input, and means for setting a destination by voice input. This enables elderly people to easily set a destination and send a message by voice input in an autonomous vehicle.

[0535] A "means for accepting input from a user" is an interface for receiving user input in the form of voice or text.

[0536] The "means for analyzing user input and inquiring about the destination and message content" is a processing means for analyzing the input from the user and identifying the necessary information (destination and message content).

[0537] The "generative artificial intelligence means for generating a message based on the destination and message content" is an artificial intelligence technology for generating an appropriate message based on the destination and message content designated by the user.

[0538] The "means for confirming the generated message with the user" is a means for presenting the contents of the generated message to the user and requesting confirmation.

[0539] The "means for sending a message after user confirmation" is a means for sending a message that has been confirmed by the user to a specified destination.

[0540] The "means by which the user terminal accepts voice input" refers to the means by which the user terminal recognizes voice and receives the content of the voice.

[0541] The "means for setting a destination by voice input" is a means for analyzing a voice input and setting a destination for the vehicle.

[0542] The system for carrying out the present invention accepts input from a user, creates a message based on the input, sets a destination, and sends the message. A method for realizing this system will be described in detail below.

[0543] System Configuration

[0544] This system is broadly composed of the following components:

[0545] 1. User Device

[0546] 2. Server

[0547] 3. Means of communication

[0548] User terminal

[0549] The user terminal is a smartphone or tablet used by the elderly, equipped with a microphone for accepting voice input, and with software for performing voice recognition (e.g., a speech_recognition library) installed.

[0550] server

[0551] The server analyzes the user's input and identifies the recipient and message content. Specifically, it uses voice recognition technology to convert the user's voice input into text data, and then uses natural language processing technology to extract the recipient and message content.

[0552] The server has the following main functions:

[0553] 1. Speech recognition technology: Use the speech_recognition library to convert voice data into text data.

[0554] 2. Natural language processing technology: Generative AI models are used to generate appropriate messages from user voice input, such as instructions for setting a destination or extracting message content.

[0555] 3. Message generation: Create a message based on the generated text data and ask the user for confirmation.

[0556] 4. Send message: Once confirmed, the message is sent using a communication method such as the LINE API.

[0557] communication means

[0558] The Internet and mobile networks are used as communication means, enabling data communication between the server and user terminals, and between message destinations.

[0559] Specific examples

[0560] Prompt Sentence Examples

[0561] Consider the case where a user speaks the following:

[0562] "Please let my daughter know her doctor's appointment time next week."

[0563] 1. Voice input

[0564] The user inputs voice commands in the car such as, "Please tell my daughter when she will go to the hospital next week." The user terminal receives this voice command and collects the voice data through the microphone.

[0565] 2. Data Analysis and Transformation

[0566] The user device sends the collected voice data to the server, which uses the speech_recognition library to convert the voice data into text data and then uses natural language processing technology to extract the recipient ("daughter") and the message content ("Please let me know her doctor's appointment time next week").

[0567] 3. Message Creation

[0568] Based on the generative artificial intelligence model, the server generates an appropriate message such as "Dear daughter: Please let me know when she will be coming to the hospital next week." The generated message is then sent to the user's device, where the user is asked to confirm it.

[0569] 4. Sending a message

[0570] Once the user has confirmed the details, the server will send the message to the daughter using the LINE API, which allows for smooth destination setting and message sending.

[0571] In this way, the system of the present invention is designed to enable elderly people to easily use advanced technology, allowing users to use self-driving vehicles with peace of mind and send necessary messages accurately.

[0572] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0573] Step 1:

[0574] The user inputs voice and the device receives the voice. The voice input uses a prompt such as "Please tell my daughter when she will be going to the doctor next week." The device uses a microphone to collect the voice data and temporarily stores it.

[0575] Input: Voice input from the user

[0576] Output: Audio data

[0577] Step 2:

[0578] The device sends the collected voice data to a server, where it uploads the voice data to the server via the Internet or a mobile network.

[0579] Input: Audio data

[0580] Output: Audio data sent to the server

[0581] Step 3:

[0582] The server analyzes the received voice data and converts it into text data. Specifically, it uses the speech_recognition library to convert the voice into text. This process results in the text data "Please let my daughter know when she will go to the hospital next week."

[0583] Input: Audio data

[0584] Output: Text data

[0585] Step 4:

[0586] The server analyzes the text data and extracts the recipient and message content. Using a generative AI model, it identifies "daughter" as the recipient and "let me know when she'll be going to the hospital next week" as the message content.

[0587] Input: Text data

[0588] Output: destination and message content

[0589] Step 5:

[0590] The server generates an appropriate message based on the recipient and message content, for example, "To your daughter: Please let me know when she will be coming to the hospital next week."

[0591] Input: Recipient and message content

[0592] Output: The generated message

[0593] Step 6:

[0594] The server sends the generated message to the user terminal for confirmation, and the terminal displays the message on its screen for the user to confirm.

[0595] Input: The generated message

[0596] Output: Message sent to the user's terminal

[0597] Step 7:

[0598] The user confirms the displayed message and instructs the sending. When the user presses the confirmation button, the terminal sends the instruction to the server.

[0599] Input: User confirmation prompt

[0600] Output: Confirmation instructions sent to the server

[0601] Step 8:

[0602] After the server confirms the user's identity, it sends the message to the specified destination using the LINE API, etc. The communication method is the internet or mobile network.

[0603] Input: Acknowledged message

[0604] Output: Message sent to destination

[0605] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0606] The present invention relates to a system that allows elderly people to easily send messages using digital devices. In particular, the system incorporates an emotion engine that recognizes the user's emotions and can adjust the content of messages according to the user's emotions.

[0607] 1. System Configuration

[0608] This system consists of the following main components:

[0609] User devices: Devices such as smartphones and tablets used by seniors.

[0610] Server: The central control unit that handles the main processing of input analysis, emotion recognition, message generation, and transmission.

[0611] Communication means: The Internet or mobile network to connect the server, user device, and destination.

[0612] Emotion engine: Software capable of recognizing emotions from user input and adjusting processing accordingly.

[0613] 2. User Input

[0614] The user device has an interface for accepting voice or text input. The user starts a conversation with the system through the LINE app. At that time, the user uses voice or text input to indicate that they want to send a message to their family.

[0615] 3. Parsing the input by the server

[0616] The server analyzes the input sent by the user. This analysis uses voice recognition technology and natural language processing technology. For example, if the user instructs the server to "send a message to my son," the server recognizes that it needs to prompt the user to enter the recipient ("son") and the message content.

[0617] 4. Emotion Recognition by Emotion Engine

[0618] The server uses an emotion engine to recognize emotions from the voice and text input by the user. The emotion engine analyzes the emotional characteristics of the user's speech and text and identifies emotions such as joy, sadness, and anger.

[0619] 5. Specify recipient and message content

[0620] The server asks the user to choose a recipient and a message content. If the user selects "my son" as the recipient and enters "Let's go for a walk together next Sunday," the server creates a message based on this information.

[0621] 6. Message Generation Using Generative AI

[0622] The server uses generative artificial intelligence technology to generate an appropriate message based on the user's input and the emotion recognition results. The message content is adjusted according to the analysis results of the emotion engine. For example, if the user is feeling happy, the server generates a message such as "To your son: I'm so happy! Let's take a walk together next Sunday."

[0623] 7. User Confirmation

[0624] The server presents the generated message to the user and asks for confirmation. If the user confirms and instructs transmission, the server executes the message transmission process.

[0625] 8. Server-sent messages

[0626] The server then uses an appropriate communication method to send the generated message to the specified destination, using the LINE API or similar to actually send the message.

[0627] 9. Examples:

[0628] Consider the case where a user opens the LINE app and says, "Tell my grandson happy birthday." The user's device receives this input as voice and sends the data to the server. The server converts the voice into text, analyzes it, and identifies the recipient ("grandson") and the message content ("Happy Birthday"). Next, the emotion engine recognizes the user's emotion of joy. After that, the generative AI generates a message saying, "To my grandson: Happy birthday! I'm so happy," and asks the user for confirmation. Once the user has confirmed, the server actually sends the message via the LINE API.

[0629] In this way, the present invention provides a support system that enables elderly people to easily send messages, and by combining it with an emotion engine, it becomes possible to send more personalized messages.

[0630] The processing flow will be explained below.

[0631] Step 1:

[0632] The user launches the LINE app and starts chatting with the AI ​​concierge. The user requests to send a message by voice or text input. For example, the user might type, "I want to send a message to my family."

[0633] Step 2:

[0634] The terminal accepts the user's voice or text input and sends it to the server.

[0635] Step 3:

[0636] The server analyzes the user's voice input and extracts the necessary data (recipient and message content). Speech recognition technology and natural language processing technology are used for the analysis. For example, it might extract "I want to send a message to my family."

[0637] Step 4:

[0638] The server runs an emotion engine to recognize emotions from the user's voice and text input, for example, recognizing "joy" from the user's tone of voice and the content of the text.

[0639] Step 5:

[0640] The server generates a message asking for the recipient and sends it to the terminal. For example, the message "Who do you want to send a message to?" is displayed on the user's terminal.

[0641] Step 6:

[0642] The user specifies the recipient by voice or text, for example, typing "To my son."

[0643] Step 7:

[0644] The terminal sends the user's input to the server, which analyzes the input and identifies the destination.

[0645] Step 8:

[0646] The server generates a message asking for the message content and sends it to the terminal. For example, the message "What message would you like to send to your son?" is displayed on the user terminal.

[0647] Step 9:

[0648] The user inputs the message content by voice or text, for example, "Let's go for a walk together this Sunday."

[0649] Step 10:

[0650] The terminal sends the user's input to the server, which analyzes the input and obtains the message content.

[0651] Step 11:

[0652] The server uses generative artificial intelligence to generate messages based on the recipient and message content. It also adjusts the message content based on the analysis results of the emotion engine. For example, it might generate a message like, "Dear son: How are you? I'm so happy! Let's take a walk together this Sunday."

[0653] Step 12:

[0654] The server generates a message for the user to confirm the generated message and sends it to the terminal. For example, a confirmation message such as "Would you like to send the following message to your son? 'To your son: How are you? I'm so happy! Let's go for a walk together next Sunday'" is displayed on the user terminal.

[0655] Step 13:

[0656] The user checks the message content and instructs the sending of the message, for example, by typing "Yes, send it."

[0657] Step 14:

[0658] The terminal sends the user's confirmation input to the server, which receives the confirmation.

[0659] Step 15:

[0660] The server uses the LINE API to send a message to the specified recipient (son). The message is sent using the appropriate communication method.

[0661] Step 16:

[0662] The server checks whether the message was sent successfully.

[0663] Step 17:

[0664] The server generates a message notifying the user of the result of the transmission and sends it to the terminal. For example, a message saying "The message has been sent" is displayed on the user terminal.

[0665] The above steps make it possible for users to easily create messages, adjust them according to their emotions, and send them.

[0666] Example 2

[0667] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0668] For seniors, the process of sending messages using digital devices can be complicated, and sending comments that don't reflect their emotions can be stressful. Furthermore, conventional messaging systems lack the ability to analyze users' emotions and tailor messages accordingly. Therefore, the present invention aims to provide a system that allows seniors to send messages more easily and that appropriately reflect their emotions.

[0669] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0670] In this invention, the server includes means for accepting input from a user, means for analyzing the user input and inquiring about the recipient and message content, means for recognizing emotions, generative artificial intelligence means for generating a message based on the recipient, message content, and emotion recognition results, means for confirming the generated message with the user, and means for sending the message after the user's confirmation. This makes it possible for elderly people to easily use digital devices and send messages that reflect their emotions.

[0671] "Means for accepting input from the user" refers to the function of receiving data on a digital device when a user inputs voice or text.

[0672] The "means for analyzing user input and inquiring about the recipient and message content" is a function that analyzes the information entered by the user and asks the user to confirm the recipient and content of the message.

[0673] "Means for recognizing emotions" refers to a function for identifying emotions from a user's voice or text data, and is a technology for analyzing the user's emotional state.

[0674] "Generative AI means" refers to an AI technology that automatically creates appropriate and personalized messages based on user input and emotion recognition results.

[0675] The "means for having the user confirm the generated message" is a function for displaying the generated message content to the user, requesting confirmation, and accepting instructions for correction or transmission.

[0676] The "means for sending a message after confirmation by the user" is a function for sending a message confirmed by the user to a specified destination using an appropriate communication means.

[0677] The embodiments of the present invention will be specifically described below.

[0678] This system is designed to enable elderly people to easily send messages using digital devices. The main components of the system are a user terminal, a server, a communication method, and an emotion engine.

[0679] The user devices are devices such as smartphones and tablets used by the elderly, and are equipped with an interface that accepts voice and text input. Users can access the system through the LINE app and send messages by voice input.

[0680] The server serves as the central control unit for the system, handling key processes such as input analysis, emotion recognition, message generation, and transmission. It uses Google Cloud Speech-to-Text API for voice recognition and Microsoft Azure Text Analytics emotion analysis for emotion recognition. It also uses generative artificial intelligence such as OpenAI GPT-4 for message generation.

[0681] Specifically, when a user opens the LINE app and says, "Tell my grandson happy birthday," the user's device accepts this voice input. The voice data is sent over the Internet to a server, which then converts it into text using the Google Cloud Speech-to-Text API.

[0682] The server then inputs the text data into Microsoft Azure Text Analytics Sentiment Analysis to recognize the user's emotion (in this case, joy). The server then uses OpenAI GPT-4 to generate an appropriate message based on the emotion recognition results. For example, the message generated might be, "Dear Grandson: Happy Birthday! I'm so happy."

[0683] The generated message is sent via the LINE app, and if the user confirms and instructs it to be sent, the server actually sends the message using the LINE API.

[0684] Examples of prompt sentences include the following:

[0685] "Please send a birthday message to my grandson."

[0686] "Tell my son I want him to come over."

[0687] In this way, the system allows seniors to easily send personalized messages that appropriately reflect their emotions.

[0688] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0689] Step 1:

[0690] Users launch the LINE app on their smartphone or tablet and use voice or text input.

[0691] Specifically, the user presses the voice input button on the LINE app and says, "Tell my grandson happy birthday."

[0692] As input, the user's voice data is taken into the terminal.

[0693] Step 2:

[0694] The device temporarily stores the voice data and transmits it to a server via the Internet.

[0695] Specifically, the terminal sends voice data to the specified URL of the server. The voice data is sent to the server as input.

[0696] Step 3:

[0697] The server converts the received voice data into text data using the Google Cloud Speech-to-Text API.

[0698] As input, speech data is given, and as output, text data such as "Tell my grandson happy birthday" is obtained.

[0699] The specific operation of the server is to call the Google Cloud Speech-to-Text API and convert the voice data into text.

[0700] Step 4:

[0701] The server analyzes the text data and extracts the information necessary to verify the recipient and message content.

[0702] As input, text data "Tell my grandchild happy birthday" is given, and as output, the recipient "grandchild" and the message content "Happy Birthday" are extracted.

[0703] Specifically, the server uses natural language processing technology to analyze the text and extract the required information.

[0704] Step 5:

[0705] The server inputs the text data into Microsoft Azure Text Analytics sentiment analysis to recognize the user's emotions.

[0706] The input is the text data "Tell my grandson happy birthday," and the output is the emotion "joy."

[0707] Specifically, the server calls the emotion engine and recognizes emotions from text data.

[0708] Step 6:

[0709] The server generates an appropriate message using generative artificial intelligence (OpenAI GPT-4) based on the recipient, message content, and emotion recognition results.

[0710] The inputs are given as the recipient "grandchild", the message content "Happy Birthday", and the emotion "Joy", and the output is generated as the message "To Grandchild: Happy Birthday! I'm so happy."

[0711] Specifically, the server calls the generative artificial intelligence and generates an appropriate message.

[0712] Step 7:

[0713] The server returns the generated message to the user's LINE app and presents it to them for confirmation.

[0714] The generated message "To my grandson: Happy birthday! I'm so happy" is given as input, and confirmation is requested from the user as output.

[0715] Specifically, the server uses the LINE API to send a message to the user's LINE app and displays a confirmation button.

[0716] Step 8:

[0717] The user checks the message on the LINE app and instructs it to be sent.

[0718] The message generated as input is displayed and the user presses the send confirmation button.

[0719] As a specific operation, the user presses the transmission confirmation button to issue a transmission instruction.

[0720] Step 9:

[0721] After the user confirms, the server uses the LINE API to send the message to the specified destination.

[0722] As input, the destination "Grandchild" and the generated message "To Grandchild: Happy Birthday! I'm so happy" are given, and as output the message is actually sent.

[0723] Specifically, the server calls the LINE API and sends the message to the destination.

[0724] (Application example 2)

[0725] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0726] When elderly people use food delivery services, they face the problem of difficulty in ordering due to the complexity of the operation. In addition, they face the problem of low user satisfaction due to the inability to provide personalized orders that take into account the user's emotions. A system that can solve these problems is needed.

[0727] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0728] In this invention, the server includes means for accepting input from a user, means for analyzing the user input and inquiring about the recipient and message content, generative artificial intelligence means for generating a message based on the recipient and message content, means for confirming the generated message with the user, means for sending the message after the user confirms, emotion recognition means for analyzing the emotion of the user input, and means for adjusting the message content based on the emotion recognition result. This not only enables elderly people to easily order food delivery, but also enables personalized order content based on their emotions.

[0729] The "means for accepting input from a user" is an interface for obtaining input data from a user in the form of voice or text.

[0730] The "means for analyzing user input and inquiring about the destination and message content" is a processing device for analyzing user input, extracting necessary information, and confirming the destination and message content.

[0731] "Generative artificial intelligence means for generating messages based on the recipient and message content" refers to artificial intelligence technology for automatically generating appropriate messages based on user input data and emotion recognition results.

[0732] The "means for confirming the generated message to the user" is an interface for displaying the generated message to the user and requesting confirmation of the contents.

[0733] The "means for sending a message after confirmation by the user" is a communication means for sending a message confirmed by the user to a specified destination.

[0734] The "emotion recognition means for analyzing the emotion of the user input" is an analysis engine for identifying the emotion from the voice or text input by the user.

[0735] The "means for adjusting the message content based on the emotion recognition result" is a processing device for appropriately changing the content and expression of the message based on the recognized emotion.

[0736] This invention is a system that allows elderly people to easily order food delivery, and can analyze the user's emotions to provide personalized ordering. The system consists of the following main components:

[0737] 1. A means of accepting input from the user

[0738] The user terminal accepts input from the user either by voice input or text input. This is typically a smartphone or tablet, and these devices have built-in microphones.

[0739] 2. A way to parse user input and query the recipient and message content

[0740] The server first analyzes the voice or text sent by the user. In the case of voice input, speech recognition technology (for example, the speech_recognition library) is used to convert the voice to text. Then, natural language processing technology is used to identify the recipient and message content.

[0741] 3. A method for adjusting message content based on emotion recognition results

[0742] The server analyzes the data entered by the user and identifies the user's emotion using an emotion recognition engine (e.g., emotion_recognition library). Based on the recognized emotion, the server adjusts the message content.

[0743] 4. Generative AI means for generating messages

[0744] The server generates appropriate messages based on user input and emotion recognition results, using a generative AI model, and the generated messages are personalized based on the user's emotional state.

[0745] 5. A means of confirming generated messages to the user

[0746] The server presents the generated message to the user and asks for confirmation, and if the user confirms and approves the sending, the message is actually sent.

[0747] 6. Means of sending messages

[0748] The server then sends the confirmed message to the specified destination using a communication method (e.g., the Internet or a mobile network) using a food delivery API.

[0749] For example, a user might say, "Tempura udon, please." The server converts this speech into text and analyzes it. The emotion recognition engine then identifies the emotion "hurry." Based on this information, the server responds to the user, saying, "It seems you're in a hurry. Here are some recommended express menu items."

[0750] Example prompts for generative AI models

[0751] For example, if a user says, "Tempura udon, please," we convert the speech input to text and use the emotion engine to identify the emotion "hurry." We then respond to the user with the following: "Sounds like you're in a hurry. Here are some recommended express menu items:..."

[0752] In this way, the present invention makes it easier and more comfortable for seniors to use food delivery services and personalizes the experience.

[0753] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0754] Step 1:

[0755] The user starts the food delivery application and inputs the phrase "Tempura udon please" by voice or text. The device acquires this input data.

[0756] Input: User voice or text input

[0757] Output: Audio or text data

[0758] Step 2:

[0759] The device sends the acquired voice data to the server, which converts the voice data into text using voice recognition technology (speech_recognition library).

[0760] Input: Audio data

[0761] Output: Text data

[0762] Step 3:

[0763] The server analyzes the text data using natural language processing technology to identify the recipient and message content. The analysis results in the recipient (e.g., food delivery service) and message content (e.g., ordering tempura udon).

[0764] Input: Text data

[0765] Output: Recipient information and message content

[0766] Step 4:

[0767] The server uses an emotion recognition engine (emotion_recognition library) to analyze emotions from the user's text data. In this step, the user's emotional state, such as whether they are in a hurry or happy, is identified.

[0768] Input: Text data

[0769] Output: Emotional state

[0770] Step 5:

[0771] The server adjusts the message content based on the user's emotional state. For example, if the user is in a hurry, it generates a response such as, "It seems you're in a hurry. Here are some recommended express delivery options." Generative AI technology is used to create messages tailored to the user.

[0772] Input: Message content and emotional state

[0773] Output: Adjusted message content

[0774] Step 6:

[0775] The server presents the generated message to the user and asks for confirmation of the contents. The terminal displays this message to the user and accepts confirmation input.

[0776] Input: Adjusted message content

[0777] Output: User confirmation input

[0778] Step 7:

[0779] The user checks the message content and approves the sending. The terminal sends this confirmation data to the server.

[0780] Input: User confirmation input

[0781] Output: Confirmation data

[0782] Step 8:

[0783] The server sends the confirmed message to the specified destination, and executes the order via the food delivery API using the communication method (Internet or mobile network).

[0784] Input: Confirmation data and message content

[0785] Output: Food delivery order confirmation

[0786] This allows users, even seniors, to easily order food delivery and personalize the experience based on their emotions.

[0787] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0788] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0789] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0790] [Third embodiment]

[0791] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0792] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0793] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0794] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0795] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0796] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0797] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0798] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0799] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0800] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0801] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0802] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0803] The present invention relates to a system that enables elderly people to easily send messages using digital devices. Specific embodiments of the present invention will be described below.

[0804] 1. System Configuration

[0805] This system is broadly composed of the following components:

[0806] User devices: Devices such as smartphones and tablets used by seniors.

[0807] Server: The central control unit that handles the main processing of input analysis, message generation, and transmission.

[0808] Communication means: The Internet or mobile network to connect the server, user device, and destination.

[0809] 2. User Input

[0810] The user device has an interface for accepting voice or text input. The user starts a conversation with the system through the LINE app. At that time, the user uses voice or text input to indicate that they want to send a message to their family.

[0811] 3. Parsing the input by the server

[0812] The server analyzes the input sent by the user. This analysis uses voice recognition technology and natural language processing technology. For example, if the user instructs the server to "send a message to my son," the server recognizes that it needs to prompt the user to enter the recipient ("son") and the message content.

[0813] 4. Specify the recipient and message content

[0814] The server asks the user to choose a recipient and a message content. If the user selects "my son" as the recipient and enters "Let's go for a walk together next Sunday," the server creates a message based on this information.

[0815] 5. Message Generation Using Generative AI

[0816] The server uses generative artificial intelligence technology to generate an appropriate message based on the user's input. For example, a message such as "To your son: Let's go for a walk together this Sunday" is automatically generated.

[0817] 6. User Confirmation

[0818] The server presents the generated message to the user and asks for confirmation. If the user confirms and instructs transmission, the server executes the message transmission process.

[0819] 7. Server-sent messages

[0820] The server then uses an appropriate communication method to send the generated message to the specified destination, using the LINE API or similar to actually send the message.

[0821] 8. Examples:

[0822] Consider the case where a user opens the LINE app and says, "Tell my grandson happy birthday." The user's device receives this input as voice and sends the data to the server. The server converts the voice to text and identifies the recipient ("grandson") and the message content ("Happy Birthday"). The generative AI then generates the message "To Grandson: Happy Birthday" and asks the user for confirmation. Once the user has confirmed, the server actually sends the message via the LINE API.

[0823] Thus, the present invention provides a support system that enables elderly people to easily send messages, thereby enabling smooth communication.

[0824] The processing flow will be explained below.

[0825] Step 1:

[0826] The user launches the LINE app and starts a chat with the AI ​​concierge. The user requests to send a message by voice or text input.

[0827] Step 2:

[0828] The device accepts the user's voice or text input and sends it to the server. For example, the user may input "I want to send a message to my family."

[0829] Step 3:

[0830] The server analyzes the user's voice input and extracts the necessary data (recipient and message content) using voice recognition and natural language processing technologies.

[0831] Step 4:

[0832] The server generates a message asking for the recipient and sends it to the terminal. For example, the message "Who do you want to send a message to?" is displayed on the user's terminal.

[0833] Step 5:

[0834] The user specifies the recipient by voice or text, for example, typing "To my son."

[0835] Step 6:

[0836] The terminal sends the user's input to the server, which analyzes the input and checks the destination.

[0837] Step 7:

[0838] The server generates a message asking for the message content and sends it to the terminal. For example, the message "What message would you like to send to your son?" is displayed on the user terminal.

[0839] Step 8:

[0840] The user inputs the message content by voice or text, for example, "Let's go for a walk together this Sunday."

[0841] Step 9:

[0842] The terminal sends the user's input to the server, which analyzes the input and obtains the message content.

[0843] Step 10:

[0844] The server uses generative artificial intelligence to generate a message based on the recipient and message content, such as "To your son: Let's take a walk together this Sunday."

[0845] Step 11:

[0846] The server generates a message for the user to confirm the generated message and sends it to the terminal. For example, a confirmation message such as "Do you want to send the following message to your son? 'To your son: Let's take a walk together next Sunday'" is displayed on the user terminal.

[0847] Step 12:

[0848] The user checks the message content and instructs the sending of the message, for example, by typing "Yes, send it."

[0849] Step 13:

[0850] The terminal sends the user's confirmation input to the server, which receives the confirmation.

[0851] Step 14:

[0852] The server uses the LINE API to send a message to the specified recipient (son). The message is sent using the appropriate communication method.

[0853] Step 15:

[0854] The server checks whether the message was sent successfully.

[0855] Step 16:

[0856] The server generates a message notifying the user of the result of the transmission and sends it to the terminal. For example, a message saying "The message has been sent" is displayed on the user terminal.

[0857] The above steps make it possible for users to easily create and send messages.

[0858] Example 1

[0859] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0860] The present invention relates to a system for reducing the hassle and technical barriers that seniors face when sending messages using digital devices. Specifically, the purpose is to simplify the process by which seniors can easily create messages through voice input or text input and send them to specific recipients.

[0861] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0862] In this invention, the server includes input means for accepting input from the user, query means for analyzing the user input and querying the destination and message content, generative artificial intelligence means for generating a message based on the destination and message content, confirmation means for confirming the generated message with the user, and transmission means for transmitting the message after the user's confirmation. This provides an environment in which elderly users can easily send messages, enabling smoother communication.

[0863] "Input means for accepting input from the user" refers to an interface that allows a user to input information in voice or text format through a device such as a smartphone or tablet.

[0864] "Means for analyzing user input and inquiring about the destination and message content" refers to the process of analyzing received input data and asking the user questions to determine the destination and message content.

[0865] "Generative artificial intelligence means for generating messages based on the recipient and message content" refers to an artificial intelligence algorithm that utilizes natural language processing technology to generate an appropriate message based on user input.

[0866] The "means for confirming the generated message with the user" refers to a process of presenting the generated message to the user and receiving instructions for confirmation of the contents and corrections.

[0867] "Transmission means for transmitting a message after confirmation by the user" refers to a system function that actually transmits the message confirmed by the user to the designated destination.

[0868] "Communication Services" refers to the infrastructure for sending and receiving messages over the Internet and mobile networks.

[0869] "Voice input" refers to a method in which a user provides information by voice through a microphone.

[0870] "Text input" refers to a method in which a user provides textual information using a keyboard or touchscreen.

[0871] The present invention relates to a system that enables elderly people to easily send messages using digital devices. Specific embodiments of the present invention will be described below.

[0872] This system is broadly composed of the following components:

[0873] User devices: Devices such as smartphones and tablets used by seniors.

[0874] Server: The central control unit that handles the main processing of input analysis, message generation, and transmission.

[0875] Communication means: The Internet or mobile network to connect the server, user device, and destination.

[0876] The user terminal has an interface for accepting voice or text input. The user starts a dialogue with the system through a communication app. For example, the user may input "I want to send a message to my son" by voice.

[0877] The server first converts the voice data sent from the user's device into text using speech recognition technology. For example, it uses a speech recognition engine such as Google Cloud Speech-to-Text. It then uses natural language processing technology to analyze the user's intent, such as "I want to send a message to my son." Using a natural language processing engine such as Google Cloud Natural Language, it identifies the recipient and message content.

[0878] The server prompts the user to confirm the destination. For example, it might say, "Who would you like to send the message to?" If the user responds with "my son," the server then asks, "What would you like to say in the message?" The user might type, "Let's go for a walk together this Sunday." Based on this information, the server uses generative artificial intelligence techniques (e.g., OpenAI's GPT-3) to generate an appropriate message.

[0879] The generated message will read, "To your son: Let's go for a walk together this Sunday." This message is presented to the user for confirmation. It is displayed with the prompt, "Are you sure you want to send this message?" If the user confirms "Yes," the server will actually send the message using a communication method (for example, LINE's API).

[0880] As a concrete example, consider the case where a user opens a communication app and says, "Tell my grandson happy birthday." The user device receives this input as voice and sends the data to the server. The server converts the voice to text using Google Cloud Speech-to-Text, analyzes it using Google Cloud Natural Language, and identifies the recipient ("grandson") and the message content ("happy birthday"). Then, a generative artificial intelligence (e.g., OpenAI GPT-3) generates the message "To Grandson: Happy Birthday" and asks the user for confirmation. Once the user has confirmed, the server sends the message via the LINE API.

[0881] Thus, the present invention provides a support system that enables elderly people to easily send messages, thereby enabling smooth communication.

[0882] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0883] Step 1:

[0884] The user inputs their intention to send a message in voice or text format on their smartphone, tablet, or other device (user device). For example, they might say, "I want to send a message to my son." The device captures this input and sends it to the server as digital data.

[0885] Input: User voice or text input

[0886] Output: Input contents in digital data format

[0887] Step 2:

[0888] The server receives the voice data sent from the device and converts it into text using speech recognition technology. Specifically, it uses a speech recognition engine such as Google Cloud Speech-to-Text. It then analyzes the input using natural language processing technology. Using Google Cloud Natural Language or similar, it identifies the recipient ("son") and the message content ("I want to send a message").

[0889] Input: Audio input in digital data format

[0890] Output: Analysis results in text format

[0891] Step 3:

[0892] Based on the analysis results, the server displays a prompt on the terminal to the user to confirm the destination. It notifies the user, "Please tell us who you want to send the message to." If the user responds with "my son," it then displays a prompt, "Please tell us the content of the message you want to send."

[0893] Input: Text data processed by speech recognition and natural language processing

[0894] Output: Prompts for destination and message content

[0895] Step 4:

[0896] When a user types "Let's go for a walk together this Sunday," the server uses generative AI to prepare a prompt to generate an appropriate message, specifically using a generative AI model such as OpenAI GPT-3.

[0897] Input: The recipient and message content entered by the user

[0898] Output: Prompt for the generative AI

[0899] Step 5:

[0900] The server sends a prompt to the generative AI and receives a generated message, such as "To your son: Let's take a walk together this Sunday."

[0901] Input: prompt and generative AI model

[0902] Output: The generated message

[0903] Step 6:

[0904] The server sends the generated message to the user's terminal, asking for user confirmation. The message is displayed with the prompt "Are you sure you want to send this message?" The user confirms and replies "Yes."

[0905] Input: The generated message

[0906] Output: Confirmation prompt and confirmation result

[0907] Step 7:

[0908] The server sends the message to the specified destination using an appropriate communication method such as the LINE API. If the message is sent successfully, a confirmation message stating "The message has been sent" is displayed to the user.

[0909] Input: User confirmation result

[0910] Output: Actual message sent and confirmation message

[0911] Through the above steps, this system helps elderly people to send messages easily.

[0912] (Application example 1)

[0913] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0914] When elderly people use digital devices to set destinations or send messages, many of the operations are cumbersome and difficult to use. This problem is particularly pronounced when using autonomous vehicles, making it difficult for elderly people to use them with confidence. Furthermore, there are limited input methods for accurately conveying intended content, so accurate message generation and transmission cannot be guaranteed.

[0915] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0916] In this invention, the server includes means for accepting input from a user, means for analyzing the user input and inquiring about the destination and message content, generative artificial intelligence means for generating a message based on the destination and message content, means for confirming the generated message with the user, means for sending the message after the user's confirmation, means for the user terminal to accept voice input, and means for setting a destination by voice input. This enables elderly people to easily set a destination and send a message by voice input in an autonomous vehicle.

[0917] A "means for accepting input from a user" is an interface for receiving user input in the form of voice or text.

[0918] The "means for analyzing user input and inquiring about the destination and message content" is a processing means for analyzing the input from the user and identifying the necessary information (destination and message content).

[0919] The "generative artificial intelligence means for generating a message based on the destination and message content" is an artificial intelligence technology for generating an appropriate message based on the destination and message content designated by the user.

[0920] The "means for confirming the generated message with the user" is a means for presenting the contents of the generated message to the user and requesting confirmation.

[0921] The "means for sending a message after user confirmation" is a means for sending a message that has been confirmed by the user to a specified destination.

[0922] The "means by which the user terminal accepts voice input" refers to the means by which the user terminal recognizes voice and receives the content of the voice.

[0923] The "means for setting a destination by voice input" is a means for analyzing a voice input and setting a destination for the vehicle.

[0924] The system for carrying out the present invention accepts input from a user, creates a message based on the input, sets a destination, and sends the message. A method for realizing this system will be described in detail below.

[0925] System Configuration

[0926] This system is broadly composed of the following components:

[0927] 1. User Device

[0928] 2. Server

[0929] 3. Means of communication

[0930] User terminal

[0931] The user terminal is a smartphone or tablet used by the elderly, equipped with a microphone for accepting voice input, and with software for performing voice recognition (e.g., a speech_recognition library) installed.

[0932] server

[0933] The server analyzes the user's input and identifies the recipient and message content. Specifically, it uses voice recognition technology to convert the user's voice input into text data, and then uses natural language processing technology to extract the recipient and message content.

[0934] The server has the following main functions:

[0935] 1. Speech recognition technology: Use the speech_recognition library to convert voice data into text data.

[0936] 2. Natural language processing technology: Generative AI models are used to generate appropriate messages from user voice input, such as instructions for setting a destination or extracting message content.

[0937] 3. Message generation: Create a message based on the generated text data and ask the user for confirmation.

[0938] 4. Send message: Once confirmed, the message is sent using a communication method such as the LINE API.

[0939] communication means

[0940] The Internet and mobile networks are used as communication means, enabling data communication between the server and user terminals, and between message destinations.

[0941] Specific examples

[0942] Prompt Sentence Examples

[0943] Consider the case where a user speaks the following:

[0944] "Please let my daughter know her doctor's appointment time next week."

[0945] 1. Voice input

[0946] The user inputs voice commands in the car such as, "Please tell my daughter when she will go to the hospital next week." The user terminal receives this voice command and collects the voice data through the microphone.

[0947] 2. Data Analysis and Transformation

[0948] The user device sends the collected voice data to the server, which uses the speech_recognition library to convert the voice data into text data and then uses natural language processing technology to extract the recipient ("daughter") and the message content ("Please let me know her doctor's appointment time next week").

[0949] 3. Message Creation

[0950] Based on the generative artificial intelligence model, the server generates an appropriate message such as "Dear daughter: Please let me know when she will be coming to the hospital next week." The generated message is then sent to the user's device, where the user is asked to confirm it.

[0951] 4. Sending a message

[0952] Once the user has confirmed the details, the server will send the message to the daughter using the LINE API, which allows for smooth destination setting and message sending.

[0953] In this way, the system of the present invention is designed to enable elderly people to easily use advanced technology, allowing users to use self-driving vehicles with peace of mind and send necessary messages accurately.

[0954] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0955] Step 1:

[0956] The user inputs voice and the device receives the voice. The voice input uses a prompt such as "Please tell my daughter when she will be going to the doctor next week." The device uses a microphone to collect the voice data and temporarily stores it.

[0957] Input: Voice input from the user

[0958] Output: Audio data

[0959] Step 2:

[0960] The device sends the collected voice data to a server, where it uploads the voice data to the server via the Internet or a mobile network.

[0961] Input: Audio data

[0962] Output: Audio data sent to the server

[0963] Step 3:

[0964] The server analyzes the received voice data and converts it into text data. Specifically, it uses the speech_recognition library to convert the voice into text. This process results in the text data "Please let my daughter know when she will go to the hospital next week."

[0965] Input: Audio data

[0966] Output: Text data

[0967] Step 4:

[0968] The server analyzes the text data and extracts the recipient and message content. Using a generative AI model, it identifies "daughter" as the recipient and "let me know when she'll be going to the hospital next week" as the message content.

[0969] Input: Text data

[0970] Output: destination and message content

[0971] Step 5:

[0972] The server generates an appropriate message based on the recipient and message content, for example, "To your daughter: Please let me know when she will be coming to the hospital next week."

[0973] Input: Recipient and message content

[0974] Output: The generated message

[0975] Step 6:

[0976] The server sends the generated message to the user terminal for confirmation, and the terminal displays the message on its screen for the user to confirm.

[0977] Input: The generated message

[0978] Output: Message sent to the user's terminal

[0979] Step 7:

[0980] The user confirms the displayed message and instructs the sending. When the user presses the confirmation button, the terminal sends the instruction to the server.

[0981] Input: User confirmation prompt

[0982] Output: Confirmation instructions sent to the server

[0983] Step 8:

[0984] After the server confirms the user's identity, it sends the message to the specified destination using the LINE API, etc. The communication method is the internet or mobile network.

[0985] Input: Acknowledged message

[0986] Output: Message sent to destination

[0987] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0988] The present invention relates to a system that allows elderly people to easily send messages using digital devices. In particular, the system incorporates an emotion engine that recognizes the user's emotions and can adjust the content of messages according to the user's emotions.

[0989] 1. System Configuration

[0990] This system consists of the following main components:

[0991] User devices: Devices such as smartphones and tablets used by seniors.

[0992] Server: The central control unit that handles the main processing of input analysis, emotion recognition, message generation, and transmission.

[0993] Communication means: The Internet or mobile network to connect the server, user device, and destination.

[0994] Emotion engine: Software capable of recognizing emotions from user input and adjusting processing accordingly.

[0995] 2. User Input

[0996] The user device has an interface for accepting voice or text input. The user starts a conversation with the system through the LINE app. At that time, the user uses voice or text input to indicate that they want to send a message to their family.

[0997] 3. Parsing the input by the server

[0998] The server analyzes the input sent by the user. This analysis uses voice recognition technology and natural language processing technology. For example, if the user instructs the server to "send a message to my son," the server recognizes that it needs to prompt the user to enter the recipient ("son") and the message content.

[0999] 4. Emotion Recognition by Emotion Engine

[1000] The server uses an emotion engine to recognize emotions from the voice and text input by the user. The emotion engine analyzes the emotional characteristics of the user's speech and text and identifies emotions such as joy, sadness, and anger.

[1001] 5. Specify recipient and message content

[1002] The server asks the user to choose a recipient and a message content. If the user selects "my son" as the recipient and enters "Let's go for a walk together next Sunday," the server creates a message based on this information.

[1003] 6. Message Generation Using Generative AI

[1004] The server uses generative artificial intelligence technology to generate an appropriate message based on the user's input and the emotion recognition results. The message content is adjusted according to the analysis results of the emotion engine. For example, if the user is feeling happy, the server generates a message such as "To your son: I'm so happy! Let's take a walk together next Sunday."

[1005] 7. User Confirmation

[1006] The server presents the generated message to the user and asks for confirmation. If the user confirms and instructs transmission, the server executes the message transmission process.

[1007] 8. Server-sent messages

[1008] The server then uses an appropriate communication method to send the generated message to the specified destination, using the LINE API or similar to actually send the message.

[1009] 9. Examples:

[1010] Consider the case where a user opens the LINE app and says, "Tell my grandson happy birthday." The user's device receives this input as voice and sends the data to the server. The server converts the voice into text, analyzes it, and identifies the recipient ("grandson") and the message content ("Happy Birthday"). Next, the emotion engine recognizes the user's emotion of joy. After that, the generative AI generates a message saying, "To my grandson: Happy birthday! I'm so happy," and asks the user for confirmation. Once the user has confirmed, the server actually sends the message via the LINE API.

[1011] In this way, the present invention provides a support system that enables elderly people to easily send messages, and by combining it with an emotion engine, it becomes possible to send more personalized messages.

[1012] The processing flow will be explained below.

[1013] Step 1:

[1014] The user launches the LINE app and starts chatting with the AI ​​concierge. The user requests to send a message by voice or text input. For example, the user might type, "I want to send a message to my family."

[1015] Step 2:

[1016] The terminal accepts the user's voice or text input and sends it to the server.

[1017] Step 3:

[1018] The server analyzes the user's voice input and extracts the necessary data (recipient and message content). Speech recognition technology and natural language processing technology are used for the analysis. For example, it might extract "I want to send a message to my family."

[1019] Step 4:

[1020] The server runs an emotion engine to recognize emotions from the user's voice and text input, for example, recognizing "joy" from the user's tone of voice and the content of the text.

[1021] Step 5:

[1022] The server generates a message asking for the recipient and sends it to the terminal. For example, the message "Who do you want to send a message to?" is displayed on the user's terminal.

[1023] Step 6:

[1024] The user specifies the recipient by voice or text, for example, typing "To my son."

[1025] Step 7:

[1026] The terminal sends the user's input to the server, which analyzes the input and identifies the destination.

[1027] Step 8:

[1028] The server generates a message asking for the message content and sends it to the terminal. For example, the message "What message would you like to send to your son?" is displayed on the user terminal.

[1029] Step 9:

[1030] The user inputs the message content by voice or text, for example, "Let's go for a walk together this Sunday."

[1031] Step 10:

[1032] The terminal sends the user's input to the server, which analyzes the input and obtains the message content.

[1033] Step 11:

[1034] The server uses generative artificial intelligence to generate messages based on the recipient and message content. It also adjusts the message content based on the analysis results of the emotion engine. For example, it might generate a message like, "Dear son: How are you? I'm so happy! Let's take a walk together this Sunday."

[1035] Step 12:

[1036] The server generates a message for the user to confirm the generated message and sends it to the terminal. For example, a confirmation message such as "Would you like to send the following message to your son? 'To your son: How are you? I'm so happy! Let's go for a walk together next Sunday'" is displayed on the user terminal.

[1037] Step 13:

[1038] The user checks the message content and instructs the sending of the message, for example, by typing "Yes, send it."

[1039] Step 14:

[1040] The terminal sends the user's confirmation input to the server, which receives the confirmation.

[1041] Step 15:

[1042] The server uses the LINE API to send a message to the specified recipient (son). The message is sent using the appropriate communication method.

[1043] Step 16:

[1044] The server checks whether the message was sent successfully.

[1045] Step 17:

[1046] The server generates a message notifying the user of the result of the transmission and sends it to the terminal. For example, a message saying "The message has been sent" is displayed on the user terminal.

[1047] The above steps make it possible for users to easily create messages, adjust them according to their emotions, and send them.

[1048] Example 2

[1049] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1050] For seniors, the process of sending messages using digital devices can be complicated, and sending comments that don't reflect their emotions can be stressful. Furthermore, conventional messaging systems lack the ability to analyze users' emotions and tailor messages accordingly. Therefore, the present invention aims to provide a system that allows seniors to send messages more easily and that appropriately reflect their emotions.

[1051] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1052] In this invention, the server includes means for accepting input from a user, means for analyzing the user input and inquiring about the recipient and message content, means for recognizing emotions, generative artificial intelligence means for generating a message based on the recipient, message content, and emotion recognition results, means for confirming the generated message with the user, and means for sending the message after the user's confirmation. This makes it possible for elderly people to easily use digital devices and send messages that reflect their emotions.

[1053] "Means for accepting input from the user" refers to the function of receiving data on a digital device when a user inputs voice or text.

[1054] The "means for analyzing user input and inquiring about the recipient and message content" is a function that analyzes the information entered by the user and asks the user to confirm the recipient and content of the message.

[1055] "Means for recognizing emotions" refers to a function for identifying emotions from a user's voice or text data, and is a technology for analyzing the user's emotional state.

[1056] "Generative AI means" refers to an AI technology that automatically creates appropriate and personalized messages based on user input and emotion recognition results.

[1057] The "means for having the user confirm the generated message" is a function for displaying the generated message content to the user, requesting confirmation, and accepting instructions for correction or transmission.

[1058] The "means for sending a message after confirmation by the user" is a function for sending a message confirmed by the user to a specified destination using an appropriate communication means.

[1059] The embodiments of the present invention will be specifically described below.

[1060] This system is designed to enable elderly people to easily send messages using digital devices. The main components of the system are a user terminal, a server, a communication method, and an emotion engine.

[1061] The user devices are devices such as smartphones and tablets used by the elderly, and are equipped with an interface that accepts voice and text input. Users can access the system through the LINE app and send messages by voice input.

[1062] The server serves as the central control unit for the system, handling key processes such as input analysis, emotion recognition, message generation, and transmission. It uses Google Cloud Speech-to-Text API for voice recognition and Microsoft Azure Text Analytics emotion analysis for emotion recognition. It also uses generative artificial intelligence such as OpenAI GPT-4 for message generation.

[1063] Specifically, when a user opens the LINE app and says, "Tell my grandson happy birthday," the user's device accepts this voice input. The voice data is sent over the Internet to a server, which then converts it into text using the Google Cloud Speech-to-Text API.

[1064] The server then inputs the text data into Microsoft Azure Text Analytics Sentiment Analysis to recognize the user's emotion (in this case, joy). The server then uses OpenAI GPT-4 to generate an appropriate message based on the emotion recognition results. For example, the message generated might be, "Dear Grandson: Happy Birthday! I'm so happy."

[1065] The generated message is sent via the LINE app, and if the user confirms and instructs it to be sent, the server actually sends the message using the LINE API.

[1066] Examples of prompt sentences include the following:

[1067] "Please send a birthday message to my grandson."

[1068] "Tell my son I want him to come over."

[1069] In this way, the system allows seniors to easily send personalized messages that appropriately reflect their emotions.

[1070] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1071] Step 1:

[1072] Users launch the LINE app on their smartphone or tablet and use voice or text input.

[1073] Specifically, the user presses the voice input button on the LINE app and says, "Tell my grandson happy birthday."

[1074] As input, the user's voice data is taken into the terminal.

[1075] Step 2:

[1076] The device temporarily stores the voice data and transmits it to a server via the Internet.

[1077] Specifically, the terminal sends voice data to the specified URL of the server. The voice data is sent to the server as input.

[1078] Step 3:

[1079] The server converts the received voice data into text data using the Google Cloud Speech-to-Text API.

[1080] As input, speech data is given, and as output, text data such as "Tell my grandson happy birthday" is obtained.

[1081] The specific operation of the server is to call the Google Cloud Speech-to-Text API and convert the voice data into text.

[1082] Step 4:

[1083] The server analyzes the text data and extracts the information necessary to verify the recipient and message content.

[1084] As input, text data "Tell my grandchild happy birthday" is given, and as output, the recipient "grandchild" and the message content "Happy Birthday" are extracted.

[1085] Specifically, the server uses natural language processing technology to analyze the text and extract the required information.

[1086] Step 5:

[1087] The server inputs the text data into Microsoft Azure Text Analytics sentiment analysis to recognize the user's emotions.

[1088] The input is the text data "Tell my grandson happy birthday," and the output is the emotion "joy."

[1089] Specifically, the server calls the emotion engine and recognizes emotions from text data.

[1090] Step 6:

[1091] The server generates an appropriate message using generative artificial intelligence (OpenAI GPT-4) based on the recipient, message content, and emotion recognition results.

[1092] The inputs are given as the recipient "grandchild", the message content "Happy Birthday", and the emotion "Joy", and the output is generated as the message "To Grandchild: Happy Birthday! I'm so happy."

[1093] Specifically, the server calls the generative artificial intelligence and generates an appropriate message.

[1094] Step 7:

[1095] The server returns the generated message to the user's LINE app and presents it to them for confirmation.

[1096] The generated message "To my grandson: Happy birthday! I'm so happy" is given as input, and confirmation is requested from the user as output.

[1097] Specifically, the server uses the LINE API to send a message to the user's LINE app and displays a confirmation button.

[1098] Step 8:

[1099] The user checks the message on the LINE app and instructs it to be sent.

[1100] The message generated as input is displayed and the user presses the send confirmation button.

[1101] As a specific operation, the user presses the transmission confirmation button to issue a transmission instruction.

[1102] Step 9:

[1103] After the user confirms, the server uses the LINE API to send the message to the specified destination.

[1104] As input, the destination "Grandchild" and the generated message "To Grandchild: Happy Birthday! I'm so happy" are given, and as output the message is actually sent.

[1105] Specifically, the server calls the LINE API and sends the message to the destination.

[1106] (Application example 2)

[1107] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1108] When elderly people use food delivery services, they face the problem of difficulty in ordering due to the complexity of the operation. In addition, they face the problem of low user satisfaction due to the inability to provide personalized orders that take into account the user's emotions. A system that can solve these problems is needed.

[1109] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1110] In this invention, the server includes means for accepting input from a user, means for analyzing the user input and inquiring about the recipient and message content, generative artificial intelligence means for generating a message based on the recipient and message content, means for confirming the generated message with the user, means for sending the message after the user confirms, emotion recognition means for analyzing the emotion of the user input, and means for adjusting the message content based on the emotion recognition result. This not only enables elderly people to easily order food delivery, but also enables personalized order content based on their emotions.

[1111] The "means for accepting input from a user" is an interface for obtaining input data from a user in the form of voice or text.

[1112] The "means for analyzing user input and inquiring about the destination and message content" is a processing device for analyzing user input, extracting necessary information, and confirming the destination and message content.

[1113] "Generative artificial intelligence means for generating messages based on the recipient and message content" refers to artificial intelligence technology for automatically generating appropriate messages based on user input data and emotion recognition results.

[1114] The "means for confirming the generated message to the user" is an interface for displaying the generated message to the user and requesting confirmation of the contents.

[1115] The "means for sending a message after confirmation by the user" is a communication means for sending a message confirmed by the user to a specified destination.

[1116] The "emotion recognition means for analyzing the emotion of the user input" is an analysis engine for identifying the emotion from the voice or text input by the user.

[1117] The "means for adjusting the message content based on the emotion recognition result" is a processing device for appropriately changing the content and expression of the message based on the recognized emotion.

[1118] This invention is a system that allows elderly people to easily order food delivery, and can analyze the user's emotions to provide personalized ordering. The system consists of the following main components:

[1119] 1. A means of accepting input from the user

[1120] The user terminal accepts input from the user either by voice input or text input. This is typically a smartphone or tablet, and these devices have built-in microphones.

[1121] 2. A way to parse user input and query the recipient and message content

[1122] The server first analyzes the voice or text sent by the user. In the case of voice input, speech recognition technology (for example, the speech_recognition library) is used to convert the voice to text. Then, natural language processing technology is used to identify the recipient and message content.

[1123] 3. A method for adjusting message content based on emotion recognition results

[1124] The server analyzes the data entered by the user and identifies the user's emotion using an emotion recognition engine (e.g., emotion_recognition library). Based on the recognized emotion, the server adjusts the message content.

[1125] 4. Generative AI means for generating messages

[1126] The server generates appropriate messages based on user input and emotion recognition results, using a generative AI model, and the generated messages are personalized based on the user's emotional state.

[1127] 5. A means of confirming generated messages to the user

[1128] The server presents the generated message to the user and asks for confirmation, and if the user confirms and approves the sending, the message is actually sent.

[1129] 6. Means of sending messages

[1130] The server then sends the confirmed message to the specified destination using a communication method (e.g., the Internet or a mobile network) using a food delivery API.

[1131] For example, a user might say, "Tempura udon, please." The server converts this speech into text and analyzes it. The emotion recognition engine then identifies the emotion "hurry." Based on this information, the server responds to the user, saying, "It seems you're in a hurry. Here are some recommended express menu items."

[1132] Example prompts for generative AI models

[1133] For example, if a user says, "Tempura udon, please," we convert the speech input to text and use the emotion engine to identify the emotion "hurry." We then respond to the user with the following: "Sounds like you're in a hurry. Here are some recommended express menu items:..."

[1134] In this way, the present invention makes it easier and more comfortable for seniors to use food delivery services and personalizes the experience.

[1135] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1136] Step 1:

[1137] The user starts the food delivery application and inputs the phrase "Tempura udon please" by voice or text. The device acquires this input data.

[1138] Input: User voice or text input

[1139] Output: Audio or text data

[1140] Step 2:

[1141] The device sends the acquired voice data to the server, which converts the voice data into text using voice recognition technology (speech_recognition library).

[1142] Input: Audio data

[1143] Output: Text data

[1144] Step 3:

[1145] The server analyzes the text data using natural language processing technology to identify the recipient and message content. The analysis results in the recipient (e.g., food delivery service) and message content (e.g., ordering tempura udon).

[1146] Input: Text data

[1147] Output: Recipient information and message content

[1148] Step 4:

[1149] The server uses an emotion recognition engine (emotion_recognition library) to analyze emotions from the user's text data. In this step, the user's emotional state, such as whether they are in a hurry or happy, is identified.

[1150] Input: Text data

[1151] Output: Emotional state

[1152] Step 5:

[1153] The server adjusts the message content based on the user's emotional state. For example, if the user is in a hurry, it generates a response such as, "It seems you're in a hurry. Here are some recommended express delivery options." Generative AI technology is used to create messages tailored to the user.

[1154] Input: Message content and emotional state

[1155] Output: Adjusted message content

[1156] Step 6:

[1157] The server presents the generated message to the user and asks for confirmation of the contents. The terminal displays this message to the user and accepts confirmation input.

[1158] Input: Adjusted message content

[1159] Output: User confirmation input

[1160] Step 7:

[1161] The user checks the message content and approves the sending. The terminal sends this confirmation data to the server.

[1162] Input: User confirmation input

[1163] Output: Confirmation data

[1164] Step 8:

[1165] The server sends the confirmed message to the specified destination, and executes the order via the food delivery API using the communication method (Internet or mobile network).

[1166] Input: Confirmation data and message content

[1167] Output: Food delivery order confirmation

[1168] This allows users, even seniors, to easily order food delivery and personalize the experience based on their emotions.

[1169] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1170] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1171] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1172] [Fourth embodiment]

[1173] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1174] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1175] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1176] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1177] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1178] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1179] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1180] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1181] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1182] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1183] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1184] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1185] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1186] The present invention relates to a system that enables elderly people to easily send messages using digital devices. Specific embodiments of the present invention will be described below.

[1187] 1. System Configuration

[1188] This system is broadly composed of the following components:

[1189] User devices: Devices such as smartphones and tablets used by seniors.

[1190] Server: The central control unit that handles the main processing of input analysis, message generation, and transmission.

[1191] Communication means: The Internet or mobile network to connect the server, user device, and destination.

[1192] 2. User Input

[1193] The user device has an interface for accepting voice or text input. The user starts a conversation with the system through the LINE app. At that time, the user uses voice or text input to indicate that they want to send a message to their family.

[1194] 3. Parsing the input by the server

[1195] The server analyzes the input sent by the user. This analysis uses voice recognition technology and natural language processing technology. For example, if the user instructs the server to "send a message to my son," the server recognizes that it needs to prompt the user to enter the recipient ("son") and the message content.

[1196] 4. Specify the recipient and message content

[1197] The server asks the user to choose a recipient and a message content. If the user selects "my son" as the recipient and enters "Let's go for a walk together next Sunday," the server creates a message based on this information.

[1198] 5. Message Generation Using Generative AI

[1199] The server uses generative artificial intelligence technology to generate an appropriate message based on the user's input. For example, a message such as "To your son: Let's go for a walk together this Sunday" is automatically generated.

[1200] 6. User Confirmation

[1201] The server presents the generated message to the user and asks for confirmation. If the user confirms and instructs transmission, the server executes the message transmission process.

[1202] 7. Server-sent messages

[1203] The server then uses an appropriate communication method to send the generated message to the specified destination, using the LINE API or similar to actually send the message.

[1204] 8. Examples:

[1205] Consider the case where a user opens the LINE app and says, "Tell my grandson happy birthday." The user's device receives this input as voice and sends the data to the server. The server converts the voice to text and identifies the recipient ("grandson") and the message content ("Happy Birthday"). The generative AI then generates the message "To Grandson: Happy Birthday" and asks the user for confirmation. Once the user has confirmed, the server actually sends the message via the LINE API.

[1206] Thus, the present invention provides a support system that enables elderly people to easily send messages, thereby enabling smooth communication.

[1207] The processing flow will be explained below.

[1208] Step 1:

[1209] The user launches the LINE app and starts a chat with the AI ​​concierge. The user requests to send a message by voice or text input.

[1210] Step 2:

[1211] The device accepts the user's voice or text input and sends it to the server. For example, the user may input "I want to send a message to my family."

[1212] Step 3:

[1213] The server analyzes the user's voice input and extracts the necessary data (recipient and message content) using voice recognition and natural language processing technologies.

[1214] Step 4:

[1215] The server generates a message asking for the recipient and sends it to the terminal. For example, the message "Who do you want to send a message to?" is displayed on the user's terminal.

[1216] Step 5:

[1217] The user specifies the recipient by voice or text, for example, typing "To my son."

[1218] Step 6:

[1219] The terminal sends the user's input to the server, which analyzes the input and checks the destination.

[1220] Step 7:

[1221] The server generates a message asking for the message content and sends it to the terminal. For example, the message "What message would you like to send to your son?" is displayed on the user terminal.

[1222] Step 8:

[1223] The user inputs the message content by voice or text, for example, "Let's go for a walk together this Sunday."

[1224] Step 9:

[1225] The terminal sends the user's input to the server, which analyzes the input and obtains the message content.

[1226] Step 10:

[1227] The server uses generative artificial intelligence to generate a message based on the recipient and message content, such as "To your son: Let's take a walk together this Sunday."

[1228] Step 11:

[1229] The server generates a message for the user to confirm the generated message and sends it to the terminal. For example, a confirmation message such as "Do you want to send the following message to your son? 'To your son: Let's take a walk together next Sunday'" is displayed on the user terminal.

[1230] Step 12:

[1231] The user checks the message content and instructs the sending of the message, for example, by typing "Yes, send it."

[1232] Step 13:

[1233] The terminal sends the user's confirmation input to the server, which receives the confirmation.

[1234] Step 14:

[1235] The server uses the LINE API to send a message to the specified recipient (son). The message is sent using the appropriate communication method.

[1236] Step 15:

[1237] The server checks whether the message was sent successfully.

[1238] Step 16:

[1239] The server generates a message notifying the user of the result of the transmission and sends it to the terminal. For example, a message saying "The message has been sent" is displayed on the user terminal.

[1240] The above steps make it possible for users to easily create and send messages.

[1241] Example 1

[1242] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1243] The present invention relates to a system for reducing the hassle and technical barriers that seniors face when sending messages using digital devices. Specifically, the purpose is to simplify the process by which seniors can easily create messages through voice input or text input and send them to specific recipients.

[1244] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1245] In this invention, the server includes input means for accepting input from the user, query means for analyzing the user input and querying the destination and message content, generative artificial intelligence means for generating a message based on the destination and message content, confirmation means for confirming the generated message with the user, and transmission means for transmitting the message after the user's confirmation. This provides an environment in which elderly users can easily send messages, enabling smoother communication.

[1246] "Input means for accepting input from the user" refers to an interface that allows a user to input information in voice or text format through a device such as a smartphone or tablet.

[1247] "Means for analyzing user input and inquiring about the destination and message content" refers to the process of analyzing received input data and asking the user questions to determine the destination and message content.

[1248] "Generative artificial intelligence means for generating messages based on the recipient and message content" refers to an artificial intelligence algorithm that utilizes natural language processing technology to generate an appropriate message based on user input.

[1249] The "means for confirming the generated message with the user" refers to a process of presenting the generated message to the user and receiving instructions for confirmation of the contents and corrections.

[1250] "Transmission means for transmitting a message after confirmation by the user" refers to a system function that actually transmits the message confirmed by the user to the designated destination.

[1251] "Communication Services" refers to the infrastructure for sending and receiving messages over the Internet and mobile networks.

[1252] "Voice input" refers to a method in which a user provides information by voice through a microphone.

[1253] "Text input" refers to a method in which a user provides textual information using a keyboard or touchscreen.

[1254] The present invention relates to a system that enables elderly people to easily send messages using digital devices. Specific embodiments of the present invention will be described below.

[1255] This system is broadly composed of the following components:

[1256] User devices: Devices such as smartphones and tablets used by seniors.

[1257] Server: The central control unit that handles the main processing of input analysis, message generation, and transmission.

[1258] Communication means: The Internet or mobile network to connect the server, user device, and destination.

[1259] The user terminal has an interface for accepting voice or text input. The user starts a dialogue with the system through a communication app. For example, the user may input "I want to send a message to my son" by voice.

[1260] The server first converts the voice data sent from the user's device into text using speech recognition technology. For example, it uses a speech recognition engine such as Google Cloud Speech-to-Text. It then uses natural language processing technology to analyze the user's intent, such as "I want to send a message to my son." Using a natural language processing engine such as Google Cloud Natural Language, it identifies the recipient and message content.

[1261] The server prompts the user to confirm the destination. For example, it might say, "Who would you like to send the message to?" If the user responds with "my son," the server then asks, "What would you like to say in the message?" The user might type, "Let's go for a walk together this Sunday." Based on this information, the server uses generative artificial intelligence techniques (e.g., OpenAI's GPT-3) to generate an appropriate message.

[1262] The generated message will read, "To your son: Let's go for a walk together this Sunday." This message is presented to the user for confirmation. It is displayed with the prompt, "Are you sure you want to send this message?" If the user confirms "Yes," the server will actually send the message using a communication method (for example, LINE's API).

[1263] As a concrete example, consider the case where a user opens a communication app and says, "Tell my grandson happy birthday." The user device receives this input as voice and sends the data to the server. The server converts the voice to text using Google Cloud Speech-to-Text, analyzes it using Google Cloud Natural Language, and identifies the recipient ("grandson") and the message content ("happy birthday"). Then, a generative artificial intelligence (e.g., OpenAI GPT-3) generates the message "To Grandson: Happy Birthday" and asks the user for confirmation. Once the user has confirmed, the server sends the message via the LINE API.

[1264] Thus, the present invention provides a support system that enables elderly people to easily send messages, thereby enabling smooth communication.

[1265] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1266] Step 1:

[1267] The user inputs their intention to send a message in voice or text format on their smartphone, tablet, or other device (user device). For example, they might say, "I want to send a message to my son." The device captures this input and sends it to the server as digital data.

[1268] Input: User voice or text input

[1269] Output: Input contents in digital data format

[1270] Step 2:

[1271] The server receives the voice data sent from the device and converts it into text using speech recognition technology. Specifically, it uses a speech recognition engine such as Google Cloud Speech-to-Text. It then analyzes the input using natural language processing technology. Using Google Cloud Natural Language or similar, it identifies the recipient ("son") and the message content ("I want to send a message").

[1272] Input: Audio input in digital data format

[1273] Output: Analysis results in text format

[1274] Step 3:

[1275] Based on the analysis results, the server displays a prompt on the terminal to the user to confirm the destination. It notifies the user, "Please tell us who you want to send the message to." If the user responds with "my son," it then displays a prompt, "Please tell us the content of the message you want to send."

[1276] Input: Text data processed by speech recognition and natural language processing

[1277] Output: Prompts for destination and message content

[1278] Step 4:

[1279] When a user types "Let's go for a walk together this Sunday," the server uses generative AI to prepare a prompt to generate an appropriate message, specifically using a generative AI model such as OpenAI GPT-3.

[1280] Input: The recipient and message content entered by the user

[1281] Output: Prompt for the generative AI

[1282] Step 5:

[1283] The server sends a prompt to the generative AI and receives a generated message, such as "To your son: Let's take a walk together this Sunday."

[1284] Input: prompt and generative AI model

[1285] Output: The generated message

[1286] Step 6:

[1287] The server sends the generated message to the user's terminal, asking for user confirmation. The message is displayed with the prompt "Are you sure you want to send this message?" The user confirms and replies "Yes."

[1288] Input: The generated message

[1289] Output: Confirmation prompt and confirmation result

[1290] Step 7:

[1291] The server sends the message to the specified destination using an appropriate communication method such as the LINE API. If the message is sent successfully, a confirmation message stating "The message has been sent" is displayed to the user.

[1292] Input: User confirmation result

[1293] Output: Actual message sent and confirmation message

[1294] Through the above steps, this system helps elderly people to send messages easily.

[1295] (Application example 1)

[1296] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1297] When elderly people use digital devices to set destinations or send messages, many of the operations are cumbersome and difficult to use. This problem is particularly pronounced when using autonomous vehicles, making it difficult for elderly people to use them with confidence. Furthermore, there are limited input methods for accurately conveying intended content, so accurate message generation and transmission cannot be guaranteed.

[1298] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1299] In this invention, the server includes means for accepting input from a user, means for analyzing the user input and inquiring about the destination and message content, generative artificial intelligence means for generating a message based on the destination and message content, means for confirming the generated message with the user, means for sending the message after the user's confirmation, means for the user terminal to accept voice input, and means for setting a destination by voice input. This enables elderly people to easily set a destination and send a message by voice input in an autonomous vehicle.

[1300] A "means for accepting input from a user" is an interface for receiving user input in the form of voice or text.

[1301] The "means for analyzing user input and inquiring about the destination and message content" is a processing means for analyzing the input from the user and identifying the necessary information (destination and message content).

[1302] The "generative artificial intelligence means for generating a message based on the destination and message content" is an artificial intelligence technology for generating an appropriate message based on the destination and message content designated by the user.

[1303] The "means for confirming the generated message with the user" is a means for presenting the contents of the generated message to the user and requesting confirmation.

[1304] The "means for sending a message after user confirmation" is a means for sending a message that has been confirmed by the user to a specified destination.

[1305] The "means by which the user terminal accepts voice input" refers to the means by which the user terminal recognizes voice and receives the content of the voice.

[1306] The "means for setting a destination by voice input" is a means for analyzing a voice input and setting a destination for the vehicle.

[1307] The system for carrying out the present invention accepts input from a user, creates a message based on the input, sets a destination, and sends the message. A method for realizing this system will be described in detail below.

[1308] System Configuration

[1309] This system is broadly composed of the following components:

[1310] 1. User Device

[1311] 2. Server

[1312] 3. Means of communication

[1313] User terminal

[1314] The user terminal is a smartphone or tablet used by the elderly, equipped with a microphone for accepting voice input, and with software for performing voice recognition (e.g., a speech_recognition library) installed.

[1315] server

[1316] The server analyzes the user's input and identifies the recipient and message content. Specifically, it uses voice recognition technology to convert the user's voice input into text data, and then uses natural language processing technology to extract the recipient and message content.

[1317] The server has the following main functions:

[1318] 1. Speech recognition technology: Use the speech_recognition library to convert voice data into text data.

[1319] 2. Natural language processing technology: Generative AI models are used to generate appropriate messages from user voice input, such as instructions for setting a destination or extracting message content.

[1320] 3. Message generation: Create a message based on the generated text data and ask the user for confirmation.

[1321] 4. Send message: Once confirmed, the message is sent using a communication method such as the LINE API.

[1322] communication means

[1323] The Internet and mobile networks are used as communication means, enabling data communication between the server and user terminals, and between message destinations.

[1324] Specific examples

[1325] Prompt Sentence Examples

[1326] Consider the case where a user speaks the following:

[1327] "Please let my daughter know her doctor's appointment time next week."

[1328] 1. Voice input

[1329] The user inputs voice commands in the car such as, "Please tell my daughter when she will go to the hospital next week." The user terminal receives this voice command and collects the voice data through the microphone.

[1330] 2. Data Analysis and Transformation

[1331] The user device sends the collected voice data to the server, which uses the speech_recognition library to convert the voice data into text data and then uses natural language processing technology to extract the recipient ("daughter") and the message content ("Please let me know her doctor's appointment time next week").

[1332] 3. Message Creation

[1333] Based on the generative artificial intelligence model, the server generates an appropriate message such as "Dear daughter: Please let me know when she will be coming to the hospital next week." The generated message is then sent to the user's device, where the user is asked to confirm it.

[1334] 4. Sending a message

[1335] Once the user has confirmed the details, the server will send the message to the daughter using the LINE API, which allows for smooth destination setting and message sending.

[1336] In this way, the system of the present invention is designed to enable elderly people to easily use advanced technology, allowing users to use self-driving vehicles with peace of mind and send necessary messages accurately.

[1337] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1338] Step 1:

[1339] The user inputs voice and the device receives the voice. The voice input uses a prompt such as "Please tell my daughter when she will be going to the doctor next week." The device uses a microphone to collect the voice data and temporarily stores it.

[1340] Input: Voice input from the user

[1341] Output: Audio data

[1342] Step 2:

[1343] The device sends the collected voice data to a server, where it uploads the voice data to the server via the Internet or a mobile network.

[1344] Input: Audio data

[1345] Output: Audio data sent to the server

[1346] Step 3:

[1347] The server analyzes the received voice data and converts it into text data. Specifically, it uses the speech_recognition library to convert the voice into text. This process results in the text data "Please let my daughter know when she will go to the hospital next week."

[1348] Input: Audio data

[1349] Output: Text data

[1350] Step 4:

[1351] The server analyzes the text data and extracts the recipient and message content. Using a generative AI model, it identifies "daughter" as the recipient and "let me know when she'll be going to the hospital next week" as the message content.

[1352] Input: Text data

[1353] Output: destination and message content

[1354] Step 5:

[1355] The server generates an appropriate message based on the recipient and message content, for example, "To your daughter: Please let me know when she will be coming to the hospital next week."

[1356] Input: Recipient and message content

[1357] Output: The generated message

[1358] Step 6:

[1359] The server sends the generated message to the user terminal for confirmation, and the terminal displays the message on its screen for the user to confirm.

[1360] Input: The generated message

[1361] Output: Message sent to the user's terminal

[1362] Step 7:

[1363] The user confirms the displayed message and instructs the sending. When the user presses the confirmation button, the terminal sends the instruction to the server.

[1364] Input: User confirmation prompt

[1365] Output: Confirmation instructions sent to the server

[1366] Step 8:

[1367] After the server confirms the user's identity, it sends the message to the specified destination using the LINE API, etc. The communication method is the internet or mobile network.

[1368] Input: Acknowledged message

[1369] Output: Message sent to destination

[1370] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1371] The present invention relates to a system that allows elderly people to easily send messages using digital devices. In particular, the system incorporates an emotion engine that recognizes the user's emotions and can adjust the content of messages according to the user's emotions.

[1372] 1. System Configuration

[1373] This system consists of the following main components:

[1374] User devices: Devices such as smartphones and tablets used by seniors.

[1375] Server: The central control unit that handles the main processing of input analysis, emotion recognition, message generation, and transmission.

[1376] Communication means: The Internet or mobile network to connect the server, user device, and destination.

[1377] Emotion engine: Software capable of recognizing emotions from user input and adjusting processing accordingly.

[1378] 2. User Input

[1379] The user device has an interface for accepting voice or text input. The user starts a conversation with the system through the LINE app. At that time, the user uses voice or text input to indicate that they want to send a message to their family.

[1380] 3. Parsing the input by the server

[1381] The server analyzes the input sent by the user. This analysis uses voice recognition technology and natural language processing technology. For example, if the user instructs the server to "send a message to my son," the server recognizes that it needs to prompt the user to enter the recipient ("son") and the message content.

[1382] 4. Emotion Recognition by Emotion Engine

[1383] The server uses an emotion engine to recognize emotions from the voice and text input by the user. The emotion engine analyzes the emotional characteristics of the user's speech and text and identifies emotions such as joy, sadness, and anger.

[1384] 5. Specify recipient and message content

[1385] The server asks the user to choose a recipient and a message content. If the user selects "my son" as the recipient and enters "Let's go for a walk together next Sunday," the server creates a message based on this information.

[1386] 6. Message Generation Using Generative AI

[1387] The server uses generative artificial intelligence technology to generate an appropriate message based on the user's input and the emotion recognition results. The message content is adjusted according to the analysis results of the emotion engine. For example, if the user is feeling happy, the server generates a message such as "To your son: I'm so happy! Let's take a walk together next Sunday."

[1388] 7. User Confirmation

[1389] The server presents the generated message to the user and asks for confirmation. If the user confirms and instructs transmission, the server executes the message transmission process.

[1390] 8. Server-sent messages

[1391] The server then uses an appropriate communication method to send the generated message to the specified destination, using the LINE API or similar to actually send the message.

[1392] 9. Examples:

[1393] Consider the case where a user opens the LINE app and says, "Tell my grandson happy birthday." The user's device receives this input as voice and sends the data to the server. The server converts the voice into text, analyzes it, and identifies the recipient ("grandson") and the message content ("Happy Birthday"). Next, the emotion engine recognizes the user's emotion of joy. After that, the generative AI generates a message saying, "To my grandson: Happy birthday! I'm so happy," and asks the user for confirmation. Once the user has confirmed, the server actually sends the message via the LINE API.

[1394] In this way, the present invention provides a support system that enables elderly people to easily send messages, and by combining it with an emotion engine, it becomes possible to send more personalized messages.

[1395] The processing flow will be explained below.

[1396] Step 1:

[1397] The user launches the LINE app and starts chatting with the AI ​​concierge. The user requests to send a message by voice or text input. For example, the user might type, "I want to send a message to my family."

[1398] Step 2:

[1399] The terminal accepts the user's voice or text input and sends it to the server.

[1400] Step 3:

[1401] The server analyzes the user's voice input and extracts the necessary data (recipient and message content). Speech recognition technology and natural language processing technology are used for the analysis. For example, it might extract "I want to send a message to my family."

[1402] Step 4:

[1403] The server runs an emotion engine to recognize emotions from the user's voice and text input, for example, recognizing "joy" from the user's tone of voice and the content of the text.

[1404] Step 5:

[1405] The server generates a message asking for the recipient and sends it to the terminal. For example, the message "Who do you want to send a message to?" is displayed on the user's terminal.

[1406] Step 6:

[1407] The user specifies the recipient by voice or text, for example, typing "To my son."

[1408] Step 7:

[1409] The terminal sends the user's input to the server, which analyzes the input and identifies the destination.

[1410] Step 8:

[1411] The server generates a message asking for the message content and sends it to the terminal. For example, the message "What message would you like to send to your son?" is displayed on the user terminal.

[1412] Step 9:

[1413] The user inputs the message content by voice or text, for example, "Let's go for a walk together this Sunday."

[1414] Step 10:

[1415] The terminal sends the user's input to the server, which analyzes the input and obtains the message content.

[1416] Step 11:

[1417] The server uses generative artificial intelligence to generate messages based on the recipient and message content. It also adjusts the message content based on the analysis results of the emotion engine. For example, it might generate a message like, "Dear son: How are you? I'm so happy! Let's take a walk together this Sunday."

[1418] Step 12:

[1419] The server generates a message for the user to confirm the generated message and sends it to the terminal. For example, a confirmation message such as "Would you like to send the following message to your son? 'To your son: How are you? I'm so happy! Let's go for a walk together next Sunday'" is displayed on the user terminal.

[1420] Step 13:

[1421] The user checks the message content and instructs the sending of the message, for example, by typing "Yes, send it."

[1422] Step 14:

[1423] The terminal sends the user's confirmation input to the server, which receives the confirmation.

[1424] Step 15:

[1425] The server uses the LINE API to send a message to the specified recipient (son). The message is sent using the appropriate communication method.

[1426] Step 16:

[1427] The server checks whether the message was sent successfully.

[1428] Step 17:

[1429] The server generates a message notifying the user of the result of the transmission and sends it to the terminal. For example, a message saying "The message has been sent" is displayed on the user terminal.

[1430] The above steps make it possible for users to easily create messages, adjust them according to their emotions, and send them.

[1431] Example 2

[1432] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1433] For seniors, the process of sending messages using digital devices can be complicated, and sending comments that don't reflect their emotions can be stressful. Furthermore, conventional messaging systems lack the ability to analyze users' emotions and tailor messages accordingly. Therefore, the present invention aims to provide a system that allows seniors to send messages more easily and that appropriately reflect their emotions.

[1434] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1435] In this invention, the server includes means for accepting input from a user, means for analyzing the user input and inquiring about the recipient and message content, means for recognizing emotions, generative artificial intelligence means for generating a message based on the recipient, message content, and emotion recognition results, means for confirming the generated message with the user, and means for sending the message after the user's confirmation. This makes it possible for elderly people to easily use digital devices and send messages that reflect their emotions.

[1436] "Means for accepting input from the user" refers to the function of receiving data on a digital device when a user inputs voice or text.

[1437] The "means for analyzing user input and inquiring about the recipient and message content" is a function that analyzes the information entered by the user and asks the user to confirm the recipient and content of the message.

[1438] "Means for recognizing emotions" refers to a function for identifying emotions from a user's voice or text data, and is a technology for analyzing the user's emotional state.

[1439] "Generative AI means" refers to an AI technology that automatically creates appropriate and personalized messages based on user input and emotion recognition results.

[1440] The "means for having the user confirm the generated message" is a function for displaying the generated message content to the user, requesting confirmation, and accepting instructions for correction or transmission.

[1441] The "means for sending a message after confirmation by the user" is a function for sending a message confirmed by the user to a specified destination using an appropriate communication means.

[1442] The embodiments of the present invention will be specifically described below.

[1443] This system is designed to enable elderly people to easily send messages using digital devices. The main components of the system are a user terminal, a server, a communication method, and an emotion engine.

[1444] The user devices are devices such as smartphones and tablets used by the elderly, and are equipped with an interface that accepts voice and text input. Users can access the system through the LINE app and send messages by voice input.

[1445] The server serves as the central control unit for the system, handling key processes such as input analysis, emotion recognition, message generation, and transmission. It uses Google Cloud Speech-to-Text API for voice recognition and Microsoft Azure Text Analytics emotion analysis for emotion recognition. It also uses generative artificial intelligence such as OpenAI GPT-4 for message generation.

[1446] Specifically, when a user opens the LINE app and says, "Tell my grandson happy birthday," the user's device accepts this voice input. The voice data is sent over the Internet to a server, which then converts it into text using the Google Cloud Speech-to-Text API.

[1447] The server then inputs the text data into Microsoft Azure Text Analytics Sentiment Analysis to recognize the user's emotion (in this case, joy). The server then uses OpenAI GPT-4 to generate an appropriate message based on the emotion recognition results. For example, the message generated might be, "Dear Grandson: Happy Birthday! I'm so happy."

[1448] The generated message is sent via the LINE app, and if the user confirms and instructs it to be sent, the server actually sends the message using the LINE API.

[1449] Examples of prompt sentences include the following:

[1450] "Please send a birthday message to my grandson."

[1451] "Tell my son I want him to come over."

[1452] In this way, the system allows seniors to easily send personalized messages that appropriately reflect their emotions.

[1453] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1454] Step 1:

[1455] Users launch the LINE app on their smartphone or tablet and use voice or text input.

[1456] Specifically, the user presses the voice input button on the LINE app and says, "Tell my grandson happy birthday."

[1457] As input, the user's voice data is taken into the terminal.

[1458] Step 2:

[1459] The device temporarily stores the voice data and transmits it to a server via the Internet.

[1460] Specifically, the terminal sends voice data to the specified URL of the server. The voice data is sent to the server as input.

[1461] Step 3:

[1462] The server converts the received voice data into text data using the Google Cloud Speech-to-Text API.

[1463] As input, speech data is given, and as output, text data such as "Tell my grandson happy birthday" is obtained.

[1464] The specific operation of the server is to call the Google Cloud Speech-to-Text API and convert the voice data into text.

[1465] Step 4:

[1466] The server analyzes the text data and extracts the information necessary to verify the recipient and message content.

[1467] As input, text data "Tell my grandchild happy birthday" is given, and as output, the recipient "grandchild" and the message content "Happy Birthday" are extracted.

[1468] Specifically, the server uses natural language processing technology to analyze the text and extract the required information.

[1469] Step 5:

[1470] The server inputs the text data into Microsoft Azure Text Analytics sentiment analysis to recognize the user's emotions.

[1471] The input is the text data "Tell my grandson happy birthday," and the output is the emotion "joy."

[1472] Specifically, the server calls the emotion engine and recognizes emotions from text data.

[1473] Step 6:

[1474] The server generates an appropriate message using generative artificial intelligence (OpenAI GPT-4) based on the recipient, message content, and emotion recognition results.

[1475] The inputs are given as the recipient "grandchild", the message content "Happy Birthday", and the emotion "Joy", and the output is generated as the message "To Grandchild: Happy Birthday! I'm so happy."

[1476] Specifically, the server calls the generative artificial intelligence and generates an appropriate message.

[1477] Step 7:

[1478] The server returns the generated message to the user's LINE app and presents it to them for confirmation.

[1479] The generated message "To my grandson: Happy birthday! I'm so happy" is given as input, and confirmation is requested from the user as output.

[1480] Specifically, the server uses the LINE API to send a message to the user's LINE app and displays a confirmation button.

[1481] Step 8:

[1482] The user checks the message on the LINE app and instructs it to be sent.

[1483] The message generated as input is displayed and the user presses the send confirmation button.

[1484] As a specific operation, the user presses the transmission confirmation button to issue a transmission instruction.

[1485] Step 9:

[1486] After the user confirms, the server uses the LINE API to send the message to the specified destination.

[1487] As input, the destination "Grandchild" and the generated message "To Grandchild: Happy Birthday! I'm so happy" are given, and as output the message is actually sent.

[1488] Specifically, the server calls the LINE API and sends the message to the destination.

[1489] (Application example 2)

[1490] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1491] When elderly people use food delivery services, they face the problem of difficulty in ordering due to the complexity of the operation. In addition, they face the problem of low user satisfaction due to the inability to provide personalized orders that take into account the user's emotions. A system that can solve these problems is needed.

[1492] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1493] In this invention, the server includes means for accepting input from a user, means for analyzing the user input and inquiring about the recipient and message content, generative artificial intelligence means for generating a message based on the recipient and message content, means for confirming the generated message with the user, means for sending the message after the user confirms, emotion recognition means for analyzing the emotion of the user input, and means for adjusting the message content based on the emotion recognition result. This not only enables elderly people to easily order food delivery, but also enables personalized order content based on their emotions.

[1494] The "means for accepting input from a user" is an interface for obtaining input data from a user in the form of voice or text.

[1495] The "means for analyzing user input and inquiring about the destination and message content" is a processing device for analyzing user input, extracting necessary information, and confirming the destination and message content.

[1496] "Generative artificial intelligence means for generating messages based on the recipient and message content" refers to artificial intelligence technology for automatically generating appropriate messages based on user input data and emotion recognition results.

[1497] The "means for confirming the generated message to the user" is an interface for displaying the generated message to the user and requesting confirmation of the contents.

[1498] The "means for sending a message after confirmation by the user" is a communication means for sending a message confirmed by the user to a specified destination.

[1499] The "emotion recognition means for analyzing the emotion of the user input" is an analysis engine for identifying the emotion from the voice or text input by the user.

[1500] The "means for adjusting the message content based on the emotion recognition result" is a processing device for appropriately changing the content and expression of the message based on the recognized emotion.

[1501] This invention is a system that allows elderly people to easily order food delivery, and can analyze the user's emotions to provide personalized ordering. The system consists of the following main components:

[1502] 1. A means of accepting input from the user

[1503] The user terminal accepts input from the user either by voice input or text input. This is typically a smartphone or tablet, and these devices have built-in microphones.

[1504] 2. A way to parse user input and query the recipient and message content

[1505] The server first analyzes the voice or text sent by the user. In the case of voice input, speech recognition technology (for example, the speech_recognition library) is used to convert the voice to text. Then, natural language processing technology is used to identify the recipient and message content.

[1506] 3. A method for adjusting message content based on emotion recognition results

[1507] The server analyzes the data entered by the user and identifies the user's emotion using an emotion recognition engine (e.g., emotion_recognition library). Based on the recognized emotion, the server adjusts the message content.

[1508] 4. Generative AI means for generating messages

[1509] The server generates appropriate messages based on user input and emotion recognition results, using a generative AI model, and the generated messages are personalized based on the user's emotional state.

[1510] 5. A means of confirming generated messages to the user

[1511] The server presents the generated message to the user and asks for confirmation, and if the user confirms and approves the sending, the message is actually sent.

[1512] 6. Means of sending messages

[1513] The server then sends the confirmed message to the specified destination using a communication method (e.g., the Internet or a mobile network) using a food delivery API.

[1514] For example, a user might say, "Tempura udon, please." The server converts this speech into text and analyzes it. The emotion recognition engine then identifies the emotion "hurry." Based on this information, the server responds to the user, saying, "It seems you're in a hurry. Here are some recommended express menu items."

[1515] Example prompts for generative AI models

[1516] For example, if a user says, "Tempura udon, please," we convert the speech input to text and use the emotion engine to identify the emotion "hurry." We then respond to the user with the following: "Sounds like you're in a hurry. Here are some recommended express menu items:..."

[1517] In this way, the present invention makes it easier and more comfortable for seniors to use food delivery services and personalizes the experience.

[1518] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1519] Step 1:

[1520] The user starts the food delivery application and inputs the phrase "Tempura udon please" by voice or text. The device acquires this input data.

[1521] Input: User voice or text input

[1522] Output: Audio or text data

[1523] Step 2:

[1524] The device sends the acquired voice data to the server, which converts the voice data into text using voice recognition technology (speech_recognition library).

[1525] Input: Audio data

[1526] Output: Text data

[1527] Step 3:

[1528] The server analyzes the text data using natural language processing technology to identify the recipient and message content. The analysis results in the recipient (e.g., food delivery service) and message content (e.g., ordering tempura udon).

[1529] Input: Text data

[1530] Output: Recipient information and message content

[1531] Step 4:

[1532] The server uses an emotion recognition engine (emotion_recognition library) to analyze emotions from the user's text data. In this step, the user's emotional state, such as whether they are in a hurry or happy, is identified.

[1533] Input: Text data

[1534] Output: Emotional state

[1535] Step 5:

[1536] The server adjusts the message content based on the user's emotional state. For example, if the user is in a hurry, it generates a response such as, "It seems you're in a hurry. Here are some recommended express delivery options." Generative AI technology is used to create messages tailored to the user.

[1537] Input: Message content and emotional state

[1538] Output: Adjusted message content

[1539] Step 6:

[1540] The server presents the generated message to the user and asks for confirmation of the contents. The terminal displays this message to the user and accepts confirmation input.

[1541] Input: Adjusted message content

[1542] Output: User confirmation input

[1543] Step 7:

[1544] The user checks the message content and approves the sending. The terminal sends this confirmation data to the server.

[1545] Input: User confirmation input

[1546] Output: Confirmation data

[1547] Step 8:

[1548] The server sends the confirmed message to the specified destination, and executes the order via the food delivery API using the communication method (Internet or mobile network).

[1549] Input: Confirmation data and message content

[1550] Output: Food delivery order confirmation

[1551] This allows users, even seniors, to easily order food delivery and personalize the experience based on their emotions.

[1552] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1553] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1554] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1555] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1556] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1557] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1558] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1559] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1560] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1561] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1562] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1563] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1564] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1565] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1566] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1567] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1568] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1569] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1570] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1571] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1572] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1573] The following is further disclosed regarding the above embodiment.

[1574] (Claim 1)

[1575] means for accepting input from a user;

[1576] means for parsing user input and querying destination and message content;

[1577] A generative artificial intelligence means for generating a message based on a destination and message content;

[1578] means for confirming the generated message to a user;

[1579] means for sending the message after user confirmation;

[1580] A system including:

[1581] (Claim 2)

[1582] 2. The system according to claim 1, wherein the message sending means uses a communication service based on a sending request.

[1583] (Claim 3)

[1584] 2. The system according to claim 1, wherein the system is capable of accepting user input by either voice input or text input.

[1585] (Claim 4)

[1586] 2. The system according to claim 1, wherein the generative artificial intelligence means dynamically generates an appropriate message in response to the contents of a user's input.

[1587] "Example 1"

[1588] (Claim 1)

[1589] an input means for accepting input from a user;

[1590] query means for analyzing user input and querying destination and message content;

[1591] A generative artificial intelligence means for generating a message based on a destination and message content;

[1592] confirmation means for confirming the generated message to a user;

[1593] sending means for sending the message after user confirmation;

[1594] A communication system including:

[1595] (Claim 2)

[1596] 2. The communication system according to claim 1, wherein the message sending means uses a communication service based on a transmission request.

[1597] (Claim 3)

[1598] 2. The communication system according to claim 1, wherein the user input can be received by either voice input or text input.

[1599] "Application Example 1"

[1600] (Claim 1)

[1601] means for accepting input from a user;

[1602] means for parsing user input and querying destination and message content;

[1603] A generative artificial intelligence means for generating a message based on a destination and message content;

[1604] means for confirming the generated message to a user;

[1605] means for sending the message after user confirmation;

[1606] a means for accepting voice input in the user terminal;

[1607] A means for setting a destination by voice input;

[1608] A system including:

[1609] (Claim 2)

[1610] 2. The system according to claim 1, wherein the message sending means uses a communication service based on a sending request.

[1611] (Claim 3)

[1612] 2. The system according to claim 1, wherein the system is capable of accepting user input by either voice input or text input.

[1613] "Example 2: Combining Emotion Engines"

[1614] (Claim 1)

[1615] means for accepting input from a user;

[1616] means for parsing user input and querying destination and message content;

[1617] A means of recognizing emotions;

[1618] a generative artificial intelligence means for generating a message based on a recipient, message content, and emotion recognition results;

[1619] means for confirming the generated message to a user;

[1620] means for sending the message after user confirmation;

[1621] A system including:

[1622] (Claim 2)

[1623] 2. The system according to claim 1, wherein the message sending means uses a communication service based on a sending request.

[1624] (Claim 3)

[1625] 2. The system according to claim 1, wherein the system is capable of accepting user input by either voice input or text input.

[1626] "Application example 2 when combining emotion engines"

[1627] (Claim 1)

[1628] means for accepting input from a user;

[1629] means for parsing user input and querying destination and message content;

[1630] A generative artificial intelligence means for generating a message based on a destination and message content;

[1631] means for confirming the generated message to a user;

[1632] means for sending the message after user confirmation;

[1633] emotion recognition means for analyzing the emotion of a user input;

[1634] means for adjusting message content based on emotion recognition results;

[1635] A system including:

[1636] (Claim 2)

[1637] 2. The system according to claim 1, wherein the message sending means uses a communication service based on a sending request.

[1638] (Claim 3)

[1639] 2. The system according to claim 1, wherein the system is capable of accepting user input by either voice input or text input. [Explanation of symbols]

[1640] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for accepting input from a user; means for parsing user input and querying destination and message content; A generative artificial intelligence means for generating a message based on a destination and message content; means for confirming the generated message to a user; means for sending the message after user confirmation; A system including:

2. 2. The system according to claim 1, wherein the message sending means utilizes a communication service based on a transmission request.

3. 2. The system of claim 1, wherein the system is capable of accepting user input by either voice input or text input.

4. 2. The system according to claim 1, wherein the generative artificial intelligence means dynamically generates an appropriate message in response to the contents of a user's input.

Images