system

Abstract

Provide a system. A means for receiving complaints and worries input by a user using a mobile information terminal; natural language processing means for analyzing received input data and extracting emotions and keywords; means for generating empathy and counseling messages based on the analyzed data; a means for displaying the generated message; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, many people live with stress and worries, but there is a lack of appropriate support to alleviate these worries and distress. In particular, when people need psychological support, it is difficult to understand their worries and provide appropriate advice. Conventional counseling services are often restricted by physical location and time, making them difficult to access. For this reason, there is a need for a system that can provide quick and easy psychological support to an unspecified number of people at any time. [Means for solving the problem]

[0005] The present invention is a system that uses natural language processing technology to receive complaints and worries entered by a user using a mobile information terminal, analyze them, and extract emotions and keywords. Based on the received data, the system automatically generates empathy and counseling messages and displays appropriate support messages to the user. The system also includes a means for transmitting text data to a server, a natural language processing engine for the server to analyze the received data, and a means for dynamically generating messages based on the analysis results. This allows users to easily receive psychological support regardless of location or time, thereby reducing stress and worries.

[0006] A "mobile information terminal" is an electronic device with communication capabilities that can be carried by a user, such as a mobile phone, smartphone, or tablet.

[0007] "Complaints and worries entered by the user using a mobile information terminal" refers to text data entered by the user using the interface of the mobile information terminal, and is information relating to stress, dissatisfaction, and difficult situations in daily life.

[0008] The "receiving means" refers to a method or device for physically or electronically acquiring the text data transmitted from the mobile information terminal.

[0009] "Means for analysis" refers to natural language processing technology and its implementation means for analyzing the content of input text data and extracting emotions and keywords.

[0010] "Natural language processing means" refers to technologies and algorithms for analyzing, understanding, and generating natural language used by humans.

[0011] The "means for generating empathy and counseling messages" refers to a method or device for automatically creating messages or advice that empathize with the user's feelings based on the analysis results.

[0012] The "displaying means" refers to a display device such as a monitor or the like for visually presenting the generated message to the user.

[0013] A "server" is a computer system for processing, storing, transmitting and receiving data.

[0014] "Text data" refers to character string information entered by a user, and is data in a format that can be processed electronically.

[0015] A "natural language processing engine" is software and algorithms for analyzing text data and extracting necessary information.

[0016] "JSON format" is an abbreviation for JavaScript (registered trademark) Object Notation, and is a method for structuring data and expressing it in a lightweight, easy-to-read text format. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The present invention is a system in which a user inputs complaints or worries using a mobile information terminal, and a server analyzes the input and generates an appropriate message. Specific embodiments for carrying out the present invention will be described below.

[0039] User Input

[0040] A user starts an application using a mobile information terminal and inputs their complaints or worries. For example, they can type something like "I'm so busy at work I don't have time to rest" directly into the text input field. Then, by pressing the "Send" button, the input data is retrieved by the terminal.

[0041] Sending data

[0042] After receiving the user's input, the terminal sends the text data to the server via an HTTP POST request. The terminal sends this request and waits for a response from the server.

[0043] Data analysis

[0044] The server analyzes the text data received from the device. A natural language processing engine is used for the analysis to extract emotions and keywords from the input text. For example, for an input such as "I'm so busy at work I don't have time to rest," the analysis engine extracts "stress" as the emotion and "work," "busy," and "time to rest" as keywords.

[0045] Creating a response message

[0046] The server generates empathy and counseling messages based on the analysis results. For example, if the analysis results indicate "stress," it generates a message such as, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you." This message provides specific empathy and practical advice for the user's situation.

[0047] Sending and displaying the response

[0048] The server formats the generated message into JSON format and sends it back to the device. The device receives this response and displays it to the user. By seeing the displayed message, the user feels that their situation is understood, which may reduce their mental burden.

[0049] For example, a user who types "I'm too busy at work and don't have time to rest" may receive feedback like this:

[0050] AI: "That's really tough. I hope you can find some time to rest, even if it's just for a little while. Don't push yourself too hard, and ask for help if you need it."

[0051] This system allows users to receive psychological support relatively easily. It is also convenient because users can input their complaints and worries at any time, regardless of location or time, and receive instant feedback. This makes it possible to provide an effective means of reducing individual stress and worries and maintaining mental health.

[0052] The processing flow will be explained below.

[0053] Step 1:

[0054] A user launches an app using a mobile device. The app displays a prompt saying "Hello, how can I help you?" and provides a text input field.

[0055] Step 2:

[0056] The user enters their complaint or worry into the text input field, for example, "I'm so busy at work I don't have time to rest." Then, the user clicks the "Send" button.

[0057] Step 3:

[0058] The device obtains user input and sends the obtained text data to the server as an HTTP POST request to a pre-configured API endpoint.

[0059] Step 4:

[0060] The server receives the HTTP request, retrieves the text data, and passes it to a natural language processing engine for analysis.

[0061] Step 5:

[0062] The server uses a natural language processing engine to analyze the text data and extract emotions and keywords. For example, for an input such as "I'm so busy at work I don't have time to rest," the server extracts "stress" as the emotion and "work," "busy," and "time to rest" as keywords.

[0063] Step 6:

[0064] The server generates empathy and counseling messages based on the analysis results. For example, if the server determines that the patient is suffering from stress, it generates a message such as, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you."

[0065] Step 7:

[0066] The server formats the generated message into JSON format and sends the formatted data to the terminal as an HTTP response.

[0067] Step 8:

[0068] The device receives the response from the server, parses the received JSON data, and extracts the message.

[0069] Step 9:

[0070] The device displays the extracted message. The user confirms the displayed message. For example, a message like "AI: That's really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you" may be displayed.

[0071] The above steps realize a system that allows users to instantly receive sympathy and counseling for their complaints and worries.

[0072] Example 1

[0073] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0074] In modern society, many people suffer from stress and worries in their work and personal lives. The lack of an environment where people can easily seek advice about these worries leads to an accumulation of mental stress, creating a problem. Furthermore, time and financial constraints mean that many people are unable to receive appropriate support from professional counseling. For this reason, there is a need for a system that allows users to easily input their complaints and worries and receive feedback on them.

[0075] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0076] In this invention, the server includes means for receiving text data from a mobile information terminal, means for transmitting the received text data as an HTTP POST request, means including a natural language processing engine for analyzing the received text data, means for generating empathy and counseling messages based on the analysis results, and means for formatting the generated messages into JSON format and transmitting them again to the mobile information terminal. This allows users to easily input their complaints and worries and receive instant feedback, thereby reducing mental stress and providing psychological support.

[0077] "User" refers to an individual who uses the system to input complaints and worries and receive feedback.

[0078] "Mobile information terminal" refers to a portable computer terminal that can connect to the Internet, such as a smartphone or tablet.

[0079] "Text data" refers to text information input by a user using a mobile information terminal.

[0080] An "HTTP POST request" is one of the HTTP protocol methods for a client to send data to a server.

[0081] "Server" refers to a computer system that receives requests from clients over a network and processes and responds to the requests.

[0082] A "natural language processing engine" refers to a software system that analyzes input text data and extracts emotions and keywords.

[0083] "Emotion" refers to the user's psychological state analyzed from text data.

[0084] "Keywords" refer to important words or phrases in the text data.

[0085] A "sympathetic message" refers to a sentence that shows understanding and sympathy for the user's input.

[0086] A "counseling message" refers to a piece of text that provides advice and support for a user's concerns.

[0087] "JSON format" is an abbreviation for JavaScript Object Notation and refers to a lightweight data format for data exchange.

[0088] The present invention is a system in which a user inputs complaints or worries using a mobile information terminal, and a server analyzes the input and generates an appropriate message. Specifically, the present invention is implemented using the following hardware and software.

[0089] Provided environment

[0090] Hardware and Software

[0091] 1. Personal digital assistant: A mobile device such as a smartphone or tablet, which includes an input / output interface (e.g., a touchscreen) and internet connectivity.

[0092] 2. Server: A powerful computer system that communicates with clients over the Internet and analyzes the data it receives.

[0093] 3. Natural Language Processing Engine: A software system, such as the Google® Cloud Natural Language API, used to analyze incoming text data to extract sentiment and keywords from the text entered by the user.

[0094] 4. API Requests and Responses: HTTP POST requests and responses are used as the communication protocol for exchanging data between the mobile information terminal and the server.

[0095] Implementation method

[0096] 1. User Input

[0097] The user uses a mobile information terminal to launch a dedicated application and enter their complaints or worries into the text field. For example, they can type, "I'm so busy at work I don't have time to rest," and press the "Send" button. This operation causes the terminal to acquire the user's input data.

[0098] 2. Data transmission

[0099] The device receives the text data entered by the user and sends it to the server as an HTTP POST request, along with metadata such as the user ID and a timestamp, so the server is ready to process the data.

[0100] 3. Receiving and analyzing text data

[0101] The server receives a POST request sent from the device. The received data is decoded and passed to a natural language processing engine for sentiment analysis and keyword extraction. For example, from the text "I'm so busy at work I don't have time to rest," the emotion "stress" and the keywords "work," "busy," and "time to rest" are extracted.

[0102] 4. Creating a Response Message

[0103] The server generates empathy and counseling messages based on the analysis results. For example, if it determines that the user is experiencing stress, it generates a message such as, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you."

[0104] 5. Sending and Displaying the Response

[0105] The server formats the generated message into JSON format and sends it back to the device as an HTTP response. The device receives the response from the server, parses the received data, and displays it in the user interface. This allows the user to receive feedback and feel that their situation is understood.

[0106] Examples and prompts

[0107] If the user types "I'm too busy at work and don't have time to rest," the following feedback will be displayed:

[0108] Specific examples

[0109] User input:

[0110] "I'm so busy with work that I don't have time to rest"

[0111] Server Feedback:

[0112] "That must be really hard. I hope you can find some time to rest, even if just a little. If you need it, don't push yourself too hard and ask for help from those around you."

[0113] Prompt Sentence Examples

[0114] User input: "I'm too busy at work and I don't have time to rest."

[0115] Message to generate: A message that contains empathy and specific advice to reduce the user's stress.

[0116] In this way, we provide a system that allows users to easily input their complaints and worries on a mobile information terminal and receive appropriate feedback immediately, thereby reducing the user's mental burden and providing psychological support.

[0117] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0118] Step 1: User Input

[0119] A user starts the application using a mobile information terminal, enters their complaints or worries into the text input field, for example, "I'm so busy at work I don't have time to rest," and presses the "Send" button.

[0120] Input: User text input

[0121] Output: Text data acquired within the device when the send button is pressed

[0122] Specific behavior: The user taps the app to launch it, enters text in the text field, and presses the send button.

[0123] Step 2: Send data

[0124] The device receives the text data and sends it to the server as an HTTP POST request, which also includes metadata such as the user ID and a timestamp.

[0125] Input: Text data and metadata acquired within the device

[0126] Output: Data sent to the server as an HTTP POST request

[0127] Specific operation: The terminal stores the input data in a buffer, generates an HTTP POST request, and sends it to the server.

[0128] Step 3: Receiving text data

[0129] The server receives the HTTP POST request sent from the terminal and decodes it to extract text data and metadata.

[0130] Input: HTTP POST request from terminal

[0131] Output: Extracted text data and metadata

[0132] What it does: The server listens for HTTP requests and parses the data to get the required information.

[0133] Step 4: Analyzing the text data

[0134] The server uses a natural language processing engine to analyze the received text data and extract emotions and keywords. For example, in the case of "I'm so busy at work I don't have time to rest," the server extracts the emotion "stress" and the keywords "work," "busy," and "time to rest."

[0135] Input: Extracted text data

[0136] Output: Parsed sentiment and keywords

[0137] What happens: The server sends an API request to the natural language processing engine and runs a script to parse the results.

[0138] Step 5: Generate a response message

[0139] The server generates empathy and counseling messages based on the analysis results. For example, if the server determines that the patient is suffering from stress, it generates a message such as, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you."

[0140] Input: Parsed sentiment and keywords

[0141] Output: Generated empathy and counseling messages

[0142] Specific operation: The server executes a script that generates a message using a predefined message template based on the analysis results.

[0143] Step 6: Sending the Response

[0144] The server formats the generated message into JSON format and resends it to the terminal as an HTTP response.

[0145] Input: Generated empathy and counseling messages

[0146] Output: HTTP response in structured JSON data format

[0147] Specific operation: The server generates a message, encodes it into JSON format, and sends the data to an API endpoint that sends an HTTP response.

[0148] Step 7: Receiving and Viewing Messages

[0149] The device receives the HTTP response from the server, parses the received JSON data, and displays the parsed message in the user interface.

[0150] Input: JSON formatted HTTP response sent from the server

[0151] Output: The message that appears in the user interface

[0152] Specific behavior: The device listens for HTTP responses, parses the received data in JSON format, and displays the results in the application's UI.

[0153] These steps allow users to easily input their complaints and worries and receive instant feedback, thereby reducing mental stress and providing psychological support.

[0154] (Application example 1)

[0155] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0156] In recent years, there has been a demand for improved customer service and customer satisfaction in brick-and-mortar stores. However, due to a lack of appropriate systems for responding to minor concerns and complaints from customers, customer dissatisfaction can easily accumulate. It is also difficult to communicate effectively with customers and propose appropriate products and services.

[0157] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0158] In this invention, the server includes means for receiving complaints and worries input by a user using a mobile information terminal, natural language processing means for analyzing the received input data and extracting emotions and keywords, means for generating empathy and counseling messages based on the analyzed data, means for displaying the generated messages, means for equipping a robot with a touch screen and communicating with customers in the store, and means for proposing products and services of the store based on the customer's input. This makes it possible to provide immediate feedback on customers' worries and complaints and to propose products and services that meet the customer's needs.

[0159] A "personal digital assistant" is an electronic device that a user can carry and that allows the user to input and transmit data such as text and voice.

[0160] "Input data" refers to complaints, worries, and other text information entered by a user using a mobile information terminal or the touch screen of a robot.

[0161] "Natural language processing" refers to the technology and process for analyzing input data and extracting emotions and keywords from it.

[0162] "Empathy and counseling message" refers to a message containing empathy and advice for the user's situation, which is generated based on the analysis of data entered by the user.

[0163] "Display means" refers to techniques and devices for visually presenting the generated message to the user.

[0164] "Equipping a robot with a touchscreen" refers to attaching a touch-operable display to the exterior of the robot.

[0165] "Means of communicating with customers in-store" refers to the robot physically roaming the store and interacting with customers through touchscreens and voice.

[0166] "Means for suggesting products and services" refers to the technology and process that provides information about appropriate products and services based on customer input.

[0167] The present invention is a system for improving customer service in brick-and-mortar stores by analyzing complaints and worries entered by users using terminals, generating empathy and counseling messages, and even suggesting appropriate products and services. Specific embodiments of the present invention will be described below.

[0168] User Input

[0169] Users can use the touchscreen of the in-store robot to input their complaints and worries, for example, "I'm so busy at work I don't have time to rest." The input data is then captured by the robot.

[0170] Sending data

[0171] The text data entered by the user is sent from the robot to the server using an HTTP POST request.

[0172] Data analysis

[0173] The server analyzes the text data received from the device using a Natural Language Processing (NLP) engine (e.g., TextBlob). Through analysis, emotions and keywords from the input text are extracted. For example, for an input such as "I'm so busy at work I don't have time to rest," the emotion extracted is "stress," and the keywords extracted are "work," "busy," and "time to rest."

[0174] Creating a response message

[0175] The server generates empathy and counseling messages based on the analysis results. For example, if the analysis results indicate "stress," it generates a message such as, "That's really tough. It's important to take a break." This message shows specific empathy for the user's situation and provides practical advice.

[0176] Sending and displaying the response

[0177] The generated message is formatted in JSON format and sent back to the robot. The robot receives this message and displays it to the user. By seeing the displayed message, the user can feel that their situation is understood, which may reduce their mental burden.

[0178] Product and service suggestions

[0179] Furthermore, the system can also suggest products and services from stores related to the complaints and worries received. For example, if a user inputs that they are "too busy at work and don't have time to rest," the system can suggest products and services that have a relaxing effect.

[0180] Specific examples

[0181] Prompt Sentence Example 1:

[0182] Enter: "I'm so busy at work I don't have time to rest."

[0183] Response: "That's really tough. It's important to get some rest."

[0184] Prompt Sentence Example 2:

[0185] Enter: "I've been feeling down lately."

[0186] Response: "I get that sometimes. I'm happy to help."

[0187] The server analyzes the data entered by the user, generates optimal messages, and suggests related products and services. This system can increase customer satisfaction and improve the quality of customer service in stores.

[0188] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0189] Step 1:

[0190] The user inputs their complaints and worries into the robot's touchscreen. The user inputs the text "I'm too busy at work and I don't have time to rest" into the robot's touchscreen. The input data is captured by the terminal.

[0191] Step 2:

[0192] The device sends the user's input data to the server using an HTTP POST request. The device converts the input text data into JSON format and sends it to the server.

[0193] Step 3:

[0194] The server receives the HTTP POST request and analyzes the input data. The server uses a Natural Language Processing (NLP) engine (e.g., TextBlob) to extract emotions and keywords from the input text. The input is the text data sent by the user, and the output is the emotion "stress" and keywords such as "work," "busy," and "time to rest."

[0195] Step 4:

[0196] The server generates empathy and counseling messages based on the analysis results. Using a generative AI model, it creates a message for the emotion "stress" such as "That's really tough. It's important to take a break." The input is the analysis results, and the output is the generated empathy and counseling message.

[0197] Step 5:

[0198] The server formats the generated message in JSON format and sends it back to the device. The input is the empathy and counseling message, and the output is the message formatted in JSON format.

[0199] Step 6:

[0200] The device displays the message received from the server. Specifically, the user can see the message "This is really hard work. It's important to take a break." on the robot's touchscreen.

[0201] Step 7:

[0202] The server proposes products and services from the store that are related to the customer's complaints and worries. For example, it provides information on products (such as relaxation goods) and services (such as relaxation services) related to "stress" and "fatigue." The input is emotions and related keywords, and the output is related product proposal information.

[0203] Step 8:

[0204] The terminal displays suggested products and services to the user, who can check suggestions such as "relaxation goods" and "relaxation services" on the robot's touchscreen.

[0205] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0206] The present invention is a system that receives complaints and worries input by a user using a mobile information terminal, and a server analyzes the input and generates an appropriate message. Furthermore, by combining it with an emotion engine, the system can more accurately determine the user's emotional state and provide individual empathy and counseling messages. Specific embodiments for implementing the present invention are described below.

[0207] User Input

[0208] A user launches an app using a mobile information device. The app displays a prompt, "Hello. What's up?" and provides a text input field. The user enters their complaints or worries into the text input field. For example, they might enter, "I'm so busy at work, I don't have time to rest." Then, by pressing the "Send" button, the entered data is retrieved by the device.

[0209] Sending data

[0210] After receiving the user's input, the terminal sends the text data to the server using an HTTP POST request and waits for a response from the server.

[0211] Data analysis

[0212] The server analyzes the text data received from the device. For analysis, it uses a natural language processing engine and an emotion engine to extract emotions and keywords from the input text. For example, for an input such as "I'm so busy at work I don't have time to rest," the natural language processing engine extracts "stress" as the emotion and "work," "busy," and "time to rest" as keywords.

[0213] Emotion recognition

[0214] The server uses an emotion engine to recognize the user's specific emotional state based on the extracted keywords and context, for example, determining the level of "stress" and other related emotions (e.g., "anxiety" or "fatigue"). This allows for a more precise understanding of the user's emotional state.

[0215] Creating a response message

[0216] The server generates empathy and counseling messages based on the analysis results and the recognized emotional state. For example, if it determines that the user is experiencing "stress," it generates a message like, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you." Furthermore, if the emotion engine determines that "anxiety" or "fatigue" are also present, it generates additional messages accordingly.

[0217] Sending and displaying the response

[0218] The server formats the generated message into JSON format and sends it back to the device. The device receives this response and displays it to the user. By seeing the displayed message, the user feels that their situation is understood, which may reduce their mental burden.

[0219] For example, a user who types "I'm too busy at work and don't have time to rest" may receive feedback like this:

[0220] AI: "That's really tough. I hope you can find some time to rest, even if just a little. If you need it, don't push yourself too hard and ask for help. I'm also concerned about your anxiety. I understand that and we can try to find some solutions, little by little."

[0221] This system allows users to receive psychological support relatively easily. It is also convenient because users can input their complaints and worries at any time, regardless of location or time, and receive instant feedback. The addition of an emotion engine can further enhance this effective method of maintaining mental health by recognizing the user's emotional state in more detail and providing corresponding messages.

[0222] The processing flow will be explained below.

[0223] Step 1:

[0224] A user launches an app using a mobile device. The app displays a prompt saying "Hello, how can I help you?" and provides a text input field.

[0225] Step 2:

[0226] The user enters their complaints or worries into the text input field, for example, "I'm so busy at work I don't have time to rest," and then clicks the "Send" button.

[0227] Step 3:

[0228] The device obtains user input and sends the obtained text data to the server as an HTTP POST request to a pre-configured API endpoint.

[0229] Step 4:

[0230] The server receives the HTTP request and retrieves the text data, which is then passed to a natural language processing engine to extract sentiment and keywords.

[0231] Step 5:

[0232] The server uses a natural language processing engine to analyze the text data. For example, for an input such as "I'm so busy at work I don't have time to rest," it extracts "stress" as the emotion and "work," "busy," and "time to rest" as keywords.

[0233] Step 6:

[0234] The server uses an emotion engine to recognize the user's specific emotional state based on the extracted keywords and context, for example, determining the degree of "stress" and other related emotions (e.g., "anxiety" or "fatigue").

[0235] Step 7:

[0236] The server generates empathy and counseling messages based on the analysis results and the recognition results of the emotion engine. For example, if it determines that the symptom is "stress" and "anxiety" is also present, it generates a message saying, "That must be really tough. It would be good if you could find some time to rest. If necessary, try not to push yourself and ask for help from those around you. I'm also concerned about your anxiety. Let's understand that and try to think of ways to deal with it little by little."

[0237] Step 8:

[0238] The server formats the generated message into JSON format and sends the formatted data to the terminal as an HTTP response.

[0239] Step 9:

[0240] The device receives the response from the server, parses the received JSON data, and extracts the message.

[0241] Step 10:

[0242] The device displays the extracted message. The user confirms the displayed message. For example, a message might appear that reads, "AI: That must be really tough. I hope you can find some time to rest, even if just a little. If necessary, don't push yourself too hard and ask for help from those around you. I'm also concerned about your anxiety. I understand that, and let's try to come up with solutions little by little."

[0243] The above steps will realize a system that allows users to receive instant empathy and counseling for their complaints and worries. The addition of an emotion engine will enable a deeper understanding of the user's emotional state and provide personalized feedback accordingly.

[0244] Example 2

[0245] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0246] In modern society, mental strain caused by stress, anxiety, and constant busy schedules is a common problem for many people. However, in our busy daily lives, opportunities to appropriately share these concerns and receive professional feedback are limited. Furthermore, conventional systems have had difficulty accurately recognizing a user's emotions and providing individually tailored feedback. The present invention aims to provide a system that analyzes complaints and worries entered by a user, accurately grasps their emotional state, and provides empathy and counseling messages.

[0247] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for analyzing received input data using a natural language processing engine and an emotion engine and extracting emotions and keywords, a means for generating empathy and counseling messages based on the analyzed data, and a means for formatting the generated messages in JSON format and transmitting them to the mobile information terminal. This makes it possible to grasp the user's emotional state in more detail and provide appropriate feedback according to the individual situation.

[0248] A "personal digital assistant" is a mobile device used by a user, such as a smartphone or tablet, that is capable of text input and communication.

[0249] "Complaints and worries" are matters related to dissatisfaction, worries, stress, and difficulties that users feel in their daily lives and at work.

[0250] A "server" is a computer system that receives, analyzes, and processes data sent by users.

[0251] "Means for receiving" refers to the process or technology for transferring data from a mobile information terminal to a server, and includes, for example, communication technology using an HTTP POST request.

[0252] A "natural language processing engine" is software for analyzing text data and extracting grammatical and semantic information, and has functions such as morphological analysis and keyword extraction.

[0253] An "emotion engine" is software that analyzes emotional states from text data and recognizes the classification and intensity of emotions.

[0254] "Means for analyzing" refers to the process of processing received data using a natural language processing engine and an emotion engine to extract necessary information.

[0255] The "empathy and counseling message" is a feedback message generated according to the user's emotions and circumstances, and includes content that sympathizes with the user's feelings and provides appropriate advice and comfort.

[0256] "Generative means" refers to the processes and techniques for constructing empathy and counseling messages based on the analyzed data.

[0257] The "JSON format" is a lightweight data format for exchanging data that is easy to read for both humans and machines.

[0258] "Means for sending" refers to the process for delivering the generated message to the mobile information terminal, and includes, for example, data communication via the Internet.

[0259] "Displaying means" refers to the process or technique for visually presenting a message to a user on a mobile device.

[0260] MODE FOR CARRYING OUT THE INVENTION

[0261] This system receives complaints and worries entered by users using a mobile information terminal, and the server analyzes them to generate appropriate messages. Furthermore, by combining it with an emotion engine, the system can more accurately determine the user's emotional state and provide personalized empathy and counseling messages.

[0262] Specific Embodiments

[0263] User Input

[0264] A user launches an app using a mobile information device. The app displays a prompt, "Hello. What's up?" and provides a text input field. The user enters their complaints or worries into the text input field. For example, they might enter, "I'm so busy at work I don't have time to rest." Then, by pressing the "Send" button, the entered data is retrieved by the device.

[0265] Sending data

[0266] After receiving the user's input, the device sends the text data to the server using an HTTP POST request and waits for a response from the server. The destination URL is set in advance.

[0267] Data analysis

[0268] The server analyzes the text data received from the terminal. For the analysis, it uses a Python (registered trademark)-based natural language processing engine (e.g., spaCy) and an emotion engine (e.g., Hugging Face Transformers). It performs morphological analysis of the input text and extracts keywords. For example, in response to the input "I'm so busy at work I don't have time to rest," the natural language processing engine extracts "stress" as the emotion and "work," "busy," and "time to rest" as keywords.

[0269] Emotion recognition

[0270] The server uses an emotion engine to recognize the user's specific emotional state based on the extracted keywords and context. The emotion engine uses the BERT model to determine the level of "stress" and other related emotions (e.g., "anxiety" or "fatigue"). This process allows for a more precise understanding of the user's emotional state.

[0271] Creating a response message

[0272] The server generates empathy and counseling messages based on the analysis results and the recognized emotional state. The program uses conditional branching to select or generate appropriate messages. For example, if the condition is determined to be "stress," it generates a message saying, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help." If the emotion engine also determines that "anxiety" or "fatigue" are present, it generates additional messages accordingly.

[0273] Sending and displaying the response

[0274] The server formats the generated message into JSON format and sends it back to the device. The device receives this response and displays it to the user. The display is in text format on the device screen, and by reading the message, the user feels that their situation is understood, which may reduce their mental burden.

[0275] Examples and prompts

[0276] Below is a specific example and the prompt to input to the generative AI model.

[0277] example

[0278] User input: "I'm too busy at work and I don't have time to rest."

[0279] Generated feedback:

[0280] That must be really tough. I hope you can find some time to rest, even if just a little. If you need it, don't push yourself too hard and ask for help from those around you. I'm also concerned about your anxiety. Understand that and try to think of ways to deal with it little by little.

[0281] Prompt Sentence Examples

[0282] User input: "I'm too busy at work and I don't have time to rest."

[0283] Generative AI model input prompt:

[0284] User input: "I'm too busy at work and I don't have time to rest."

[0285] System:

[0286] The natural language processing engine analyzes the data and extracts "stress," "work," "busy," and "rest time." The emotion engine recognizes specific emotional states (e.g., stress, anxiety, fatigue) and generates appropriate messages.

[0287] This system allows users to receive psychological support relatively easily. It is also convenient because users can input their complaints and worries at any time, regardless of location or time, and receive instant feedback. The addition of an emotion engine can further enhance this effective method of maintaining mental health by recognizing the user's emotional state in more detail and providing corresponding messages.

[0288] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0289] Step 1:

[0290] A user launches an app using a mobile information device. The app displays a prompt saying "Hello. What's up?" and provides a text input field. The user enters their complaints or worries into the text input field and presses the "Send" button. Specifically, the device acquires the user's text input and prepares it for transmission. The input includes the text of the user's worries or complaints. The output is the acquired text data.

[0291] Step 2:

[0292] The device sends the user's input to the server as an HTTP POST request. This request contains the text data entered by the user. Specifically, the device sends data to the destination URL and waits for a response from the server. The input is the text data entered by the user, and the output is data sent to the server.

[0293] Step 3:

[0294] The server analyzes the text data received from the terminal. First, a natural language processing engine (e.g., spaCy) is used to perform morphological analysis and extract keywords. The text data is input for this analysis. Specifically, through morphological analysis, the keywords "work," "busy," and "time to rest" are extracted. The extracted keywords are generated as output.

[0295] Step 4:

[0296] The server uses an emotion engine (e.g., Hugging Face Transformers) to analyze the emotional state of the text data. The input is the keywords extracted in the previous step. Specifically, the emotion engine analyzes the context of the text and recognizes emotions such as "stress" and "anxiety." The output is the recognized emotional state.

[0297] Step 5:

[0298] The server generates empathy and counseling messages based on the analysis results and the recognized emotional state. The inputs are the recognized emotional state and extracted keywords. Specific operations include using conditional branching to select an appropriate message. For example, if it is determined to be "stressed," it generates a message saying, "That must be really tough. It would be good if you could find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you." The generated message is output.

[0299] Step 6:

[0300] The server formats the generated message into JSON format and sends it back to the terminal. The input is the generated message. Specifically, the generated message is converted into JSON format and an HTTP POST request is sent to the terminal again. The formatted message is generated as the output.

[0301] Step 7:

[0302] The device displays the received message to the user. The input is the received JSON format message. Specifically, the device parses the message and displays it on the screen. By reading this, the user feels that their feelings have been understood. The displayed message is generated as the output.

[0303] Through these steps, users can receive psychological support relatively easily. This system accurately grasps the user's emotional state and provides appropriate feedback to maintain mental health. It is also available at any time, regardless of location or time.

[0304] (Application example 2)

[0305] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0306] In recent years, technologies for improving consumer purchasing experiences have been attracting attention. In particular, brick-and-mortar stores are required to quickly resolve customer problems and provide appropriate support. However, current systems have difficulty accurately understanding customers' emotions and concerns and providing appropriate empathy and counseling messages. Furthermore, when using mobile information terminals to achieve this, real-time feedback and support are insufficient. This invention aims to solve these problems and quickly resolve problems customers face in brick-and-mortar stores.

[0307] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving complaints and worries input by a user using a mobile information terminal, natural language processing means for analyzing the received input data and extracting emotions and keywords, means for generating empathy and counseling messages based on the analyzed data, and means for displaying the generated messages on the mobile information terminal or the wearable display device. This makes it possible to provide immediate and appropriate support for problems users encounter in physical stores and improve customer experience.

[0308] "Mobile information terminal" refers to an electronic device that can be carried by a user and that can input and display data. Specifically, it includes smartphones and tablet terminals.

[0309] "Wearable display devices" refer to display devices that can be worn by the user. Examples include smart glasses and head-mounted displays.

[0310] "Natural language processing means" refers to technology that analyzes text data entered by users and extracts emotions and keywords. Specifically, this is done using a natural language processing engine.

[0311] "Emotion analysis engine" refers to technology that analyzes emotions contained in received text data and determines the user's emotional state, enabling a more accurate understanding of the user's emotions.

[0312] An "empathetic message" is a message that shows empathy for the user's feelings and situation and understands how the user feels, thereby reducing the user's mental burden.

[0313] A "counseling message" is a message that provides specific advice or suggestions for the problems or worries that the user is having, thereby helping the user to solve the problem.

[0314] A "server" is a computer system that receives and processes data sent by users. It uses various engines to analyze data and generate messages.

[0315] This invention is a system that receives complaints and worries input by a user using a mobile information terminal or a wearable display device, analyzes the input, and generates appropriate empathy and counseling messages. Detailed embodiments of this system are described below.

[0316] User Input

[0317] A user launches an application using a mobile information terminal (such as a smartphone or tablet) or a wearable display device (such as smart glasses). The application displays a prompt, saying, "Please let us know if you have any problems or complaints about the store," and provides a voice or text input field. For example, the user might type, "The store is too crowded and I feel uneasy. I can't even find the restroom." Then, by pressing the "Submit" button, the input data is captured by the device.

[0318] Sending data

[0319] After the terminal receives the user's input, it sends the text data to the server using an HTTP POST request and waits for a response from the server. At this time, it uses the requests module to send the data to the server.

[0320] Data analysis

[0321] The server analyzes the text data received from the device. A natural language processing engine and a sentiment analysis engine are used for the analysis, and emotions and keywords from the input text are extracted. For example, in response to an input such as "The store is too crowded and I feel uneasy. I don't know where the restroom is," the natural language processing engine extracts "stress due to crowding" as the emotion and "inside the store," "crowded," and "restroom" as keywords.

[0322] Emotion recognition

[0323] The server uses an emotion analysis engine to recognize the user's specific emotional state based on the extracted keywords and context. For example, it determines the degree of "stress due to crowding." It also determines other related emotions (e.g., "anxiety" or "dissatisfaction"), allowing for a more precise understanding of the user's emotional state.

[0324] Creating a response message

[0325] The server generates empathy and counseling messages based on the analysis results and the recognized emotional state. For example, if it determines that the customer is feeling "stressed due to crowding," it generates a message saying, "It seems you're worried about the store being crowded. The restroom is located at the back right of the store. If you wait a moment, a staff member will assist you." If the emotion analysis engine also determines that "anxiety" is present, it generates an additional message accordingly.

[0326] Sending and displaying the response

[0327] The server formats the generated message into JSON format and sends it back to the device. The device receives this response and displays it to the user. Seeing the displayed message makes the user feel that their situation is understood, which may reduce their mental burden. For example, a user who types, "The store is too crowded and I feel uneasy. I can't even find the restroom," could receive the following feedback:

[0328] Prompt Sentence Examples

[0329] User input: "The store is too crowded and I feel uneasy. I don't know where the restroom is."

[0330] Feedback message: "I understand you're concerned about the store being busy. The restroom is located at the back right of the store. Please wait a moment and a member of staff will help you."

[0331] The system allows users to receive quick support for any issues they may encounter in-store, improving their shopping experience.

[0332] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0333] Step 1:

[0334] A user launches an application using a mobile information terminal or a wearable display device. The terminal displays a prompt on the screen saying, "Please let us know if you have any problems or complaints about the store," and provides voice and text input fields. The input here is the user's complaints or worries. For example, the user might enter, "The store is too crowded and I feel uneasy. I can't even find the restroom."

[0335] input:

[0336] Text of the user's complaints or worries (e.g., "The store is too crowded and I can't relax. I don't know where the restroom is.")

[0337] output:

[0338] User input text data

[0339] Step 2:

[0340] The terminal obtains the user's input text data and sends it to the server using an HTTP POST request, using the requests module to send the data to the server.

[0341] input:

[0342] User input text data

[0343] output:

[0344] Text data sent to the server

[0345] Step 3:

[0346] The server analyzes the received text data. Using a natural language processing engine, the server extracts emotions and keywords from the input text. In this example, for the text "The store is too crowded and I feel uneasy. I don't even know where the restroom is," the server extracts "stress due to crowding" as the emotion, and extracts "inside the store," "crowded," and "restroom" as keywords.

[0347] input:

[0348] Text data sent to the server

[0349] output:

[0350] Extracted emotions and keywords

[0351] Step 4:

[0352] The server uses an emotion analysis engine to determine the user's specific emotional state based on the extracted keywords and context. For example, it evaluates the degree of "stress due to crowding" and determines related emotions. At this time, it also analyzes whether other emotions (such as "anxiety" or "dissatisfaction") are present.

[0353] input:

[0354] Extracted emotions and keywords

[0355] output:

[0356] The user's specific emotional state

[0357] Step 5:

[0358] The server generates empathy and counseling messages based on the analysis results and the recognized emotional state. For example, if it determines that the customer is feeling "stressed due to crowding," it generates a message saying, "It seems you're worried about the store being crowded. The restroom is located at the back right of the store. If you wait a moment, a staff member will assist you." If the emotion analysis engine determines that "anxiety" is also present, it generates an additional message accordingly.

[0359] input:

[0360] The user's specific emotional state

[0361] output:

[0362] Generated empathy and counseling messages

[0363] Step 6:

[0364] The server formats the generated message into JSON format and sends it back to the device. The device receives this response and displays it to the user. Seeing the displayed message can help the user feel that their situation is understood, which can reduce their mental burden.

[0365] input:

[0366] Generated empathy and counseling messages

[0367] output:

[0368] Feedback message displayed on the device

[0369] This process allows users to receive prompt and appropriate support for any issues they may have, improving their shopping experience in physical stores.

[0370] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0371] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0372] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0373] [Second embodiment]

[0374] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0375] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0376] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0377] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0378] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0379] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0380] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0381] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0382] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0383] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0384] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0385] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0386] The present invention is a system in which a user inputs complaints or worries using a mobile information terminal, and a server analyzes the input and generates an appropriate message. Specific embodiments for carrying out the present invention will be described below.

[0387] User Input

[0388] A user starts an application using a mobile information terminal and inputs their complaints or worries. For example, they can type something like "I'm so busy at work I don't have time to rest" directly into the text input field. Then, by pressing the "Send" button, the input data is retrieved by the terminal.

[0389] Sending data

[0390] After receiving the user's input, the terminal sends the text data to the server via an HTTP POST request. The terminal sends this request and waits for a response from the server.

[0391] Data analysis

[0392] The server analyzes the text data received from the device. A natural language processing engine is used for the analysis to extract emotions and keywords from the input text. For example, for an input such as "I'm so busy at work I don't have time to rest," the analysis engine extracts "stress" as the emotion and "work," "busy," and "time to rest" as keywords.

[0393] Creating a response message

[0394] The server generates empathy and counseling messages based on the analysis results. For example, if the analysis results indicate "stress," it generates a message such as, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you." This message provides specific empathy and practical advice for the user's situation.

[0395] Sending and displaying the response

[0396] The server formats the generated message into JSON format and sends it back to the device. The device receives this response and displays it to the user. By seeing the displayed message, the user feels that their situation is understood, which may reduce their mental burden.

[0397] For example, a user who types "I'm too busy at work and don't have time to rest" may receive feedback like this:

[0398] AI: "That's really tough. I hope you can find some time to rest, even if it's just for a little while. Don't push yourself too hard, and ask for help if you need it."

[0399] This system allows users to receive psychological support relatively easily. It is also convenient because users can input their complaints and worries at any time, regardless of location or time, and receive instant feedback. This makes it possible to provide an effective means of reducing individual stress and worries and maintaining mental health.

[0400] The processing flow will be explained below.

[0401] Step 1:

[0402] A user launches an app using a mobile device. The app displays a prompt saying "Hello, how can I help you?" and provides a text input field.

[0403] Step 2:

[0404] The user enters their complaint or worry into the text input field, for example, "I'm so busy at work I don't have time to rest." Then, the user clicks the "Send" button.

[0405] Step 3:

[0406] The device obtains user input and sends the obtained text data to the server as an HTTP POST request to a pre-configured API endpoint.

[0407] Step 4:

[0408] The server receives the HTTP request, retrieves the text data, and passes it to a natural language processing engine for analysis.

[0409] Step 5:

[0410] The server uses a natural language processing engine to analyze the text data and extract emotions and keywords. For example, for an input such as "I'm so busy at work I don't have time to rest," the server extracts "stress" as the emotion and "work," "busy," and "time to rest" as keywords.

[0411] Step 6:

[0412] The server generates empathy and counseling messages based on the analysis results. For example, if the server determines that the patient is suffering from stress, it generates a message such as, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you."

[0413] Step 7:

[0414] The server formats the generated message into JSON format and sends the formatted data to the terminal as an HTTP response.

[0415] Step 8:

[0416] The device receives the response from the server, parses the received JSON data, and extracts the message.

[0417] Step 9:

[0418] The device displays the extracted message. The user confirms the displayed message. For example, a message like "AI: That's really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you" may be displayed.

[0419] The above steps realize a system that allows users to instantly receive sympathy and counseling for their complaints and worries.

[0420] Example 1

[0421] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0422] In modern society, many people suffer from stress and worries in their work and personal lives. The lack of an environment where people can easily seek advice about these worries leads to an accumulation of mental stress, creating a problem. Furthermore, time and financial constraints mean that many people are unable to receive appropriate support from professional counseling. For this reason, there is a need for a system that allows users to easily input their complaints and worries and receive feedback on them.

[0423] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0424] In this invention, the server includes means for receiving text data from a mobile information terminal, means for transmitting the received text data as an HTTP POST request, means including a natural language processing engine for analyzing the received text data, means for generating empathy and counseling messages based on the analysis results, and means for formatting the generated messages into JSON format and transmitting them again to the mobile information terminal. This allows users to easily input their complaints and worries and receive instant feedback, thereby reducing mental stress and providing psychological support.

[0425] "User" refers to an individual who uses the system to input complaints and worries and receive feedback.

[0426] "Mobile information terminal" refers to a portable computer terminal that can connect to the Internet, such as a smartphone or tablet.

[0427] "Text data" refers to text information input by a user using a mobile information terminal.

[0428] An "HTTP POST request" is one of the HTTP protocol methods for a client to send data to a server.

[0429] "Server" refers to a computer system that receives requests from clients over a network and processes and responds to the requests.

[0430] A "natural language processing engine" refers to a software system that analyzes input text data and extracts emotions and keywords.

[0431] "Emotion" refers to the user's psychological state analyzed from text data.

[0432] "Keywords" refer to important words or phrases in the text data.

[0433] A "sympathetic message" refers to a sentence that shows understanding and sympathy for the user's input.

[0434] A "counseling message" refers to a piece of text that provides advice and support for a user's concerns.

[0435] "JSON format" is an abbreviation for JavaScript Object Notation and refers to a lightweight data format for data exchange.

[0436] The present invention is a system in which a user inputs complaints or worries using a mobile information terminal, and a server analyzes the input and generates an appropriate message. Specifically, the present invention is implemented using the following hardware and software.

[0437] Provided environment

[0438] Hardware and Software

[0439] 1. Personal digital assistant: A mobile device such as a smartphone or tablet, which includes an input / output interface (e.g., a touchscreen) and internet connectivity.

[0440] 2. Server: A powerful computer system that communicates with clients over the Internet and analyzes the data it receives.

[0441] 3. Natural Language Processing Engine: A software system used to analyze incoming text data, such as the Google Cloud Natural Language API, to extract sentiment and keywords from the text entered by the user.

[0442] 4. API Requests and Responses: HTTP POST requests and responses are used as the communication protocol for exchanging data between the mobile information terminal and the server.

[0443] Implementation method

[0444] 1. User Input

[0445] The user uses a mobile information terminal to launch a dedicated application and enter their complaints or worries into the text field. For example, they can type, "I'm so busy at work I don't have time to rest," and press the "Send" button. This operation causes the terminal to acquire the user's input data.

[0446] 2. Data transmission

[0447] The device receives the text data entered by the user and sends it to the server as an HTTP POST request, along with metadata such as the user ID and a timestamp, so the server is ready to process the data.

[0448] 3. Receiving and analyzing text data

[0449] The server receives a POST request sent from the device. The received data is decoded and passed to a natural language processing engine for sentiment analysis and keyword extraction. For example, from the text "I'm so busy at work I don't have time to rest," the emotion "stress" and the keywords "work," "busy," and "time to rest" are extracted.

[0450] 4. Creating a Response Message

[0451] The server generates empathy and counseling messages based on the analysis results. For example, if it determines that the user is experiencing stress, it generates a message such as, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you."

[0452] 5. Sending and Displaying the Response

[0453] The server formats the generated message into JSON format and sends it back to the device as an HTTP response. The device receives the response from the server, parses the received data, and displays it in the user interface. This allows the user to receive feedback and feel that their situation is understood.

[0454] Examples and prompts

[0455] If the user types "I'm too busy at work and don't have time to rest," the following feedback will be displayed:

[0456] Specific examples

[0457] User input:

[0458] "I'm so busy with work that I don't have time to rest"

[0459] Server Feedback:

[0460] "That must be really hard. I hope you can find some time to rest, even if just a little. If you need it, don't push yourself too hard and ask for help from those around you."

[0461] Prompt Sentence Examples

[0462] User input: "I'm too busy at work and I don't have time to rest."

[0463] Message to generate: A message that contains empathy and specific advice to reduce the user's stress.

[0464] In this way, we provide a system that allows users to easily input their complaints and worries on a mobile information terminal and receive appropriate feedback immediately, thereby reducing the user's mental burden and providing psychological support.

[0465] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0466] Step 1: User Input

[0467] A user starts the application using a mobile information terminal, enters their complaints or worries into the text input field, for example, "I'm so busy at work I don't have time to rest," and presses the "Send" button.

[0468] Input: User text input

[0469] Output: Text data acquired within the device when the send button is pressed

[0470] Specific behavior: The user taps the app to launch it, enters text in the text field, and presses the send button.

[0471] Step 2: Send data

[0472] The device receives the text data and sends it to the server as an HTTP POST request, which also includes metadata such as the user ID and a timestamp.

[0473] Input: Text data and metadata acquired within the device

[0474] Output: Data sent to the server as an HTTP POST request

[0475] Specific operation: The terminal stores the input data in a buffer, generates an HTTP POST request, and sends it to the server.

[0476] Step 3: Receiving text data

[0477] The server receives the HTTP POST request sent from the terminal and decodes it to extract text data and metadata.

[0478] Input: HTTP POST request from terminal

[0479] Output: Extracted text data and metadata

[0480] What it does: The server listens for HTTP requests and parses the data to get the required information.

[0481] Step 4: Analyzing the text data

[0482] The server uses a natural language processing engine to analyze the received text data and extract emotions and keywords. For example, in the case of "I'm so busy at work I don't have time to rest," the server extracts the emotion "stress" and the keywords "work," "busy," and "time to rest."

[0483] Input: Extracted text data

[0484] Output: Parsed sentiment and keywords

[0485] What happens: The server sends an API request to the natural language processing engine and runs a script to parse the results.

[0486] Step 5: Generate a response message

[0487] The server generates empathy and counseling messages based on the analysis results. For example, if the server determines that the patient is suffering from stress, it generates a message such as, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you."

[0488] Input: Parsed sentiment and keywords

[0489] Output: Generated empathy and counseling messages

[0490] Specific operation: The server executes a script that generates a message using a predefined message template based on the analysis results.

[0491] Step 6: Sending the Response

[0492] The server formats the generated message into JSON format and resends it to the terminal as an HTTP response.

[0493] Input: Generated empathy and counseling messages

[0494] Output: HTTP response in structured JSON data format

[0495] Specific operation: The server generates a message, encodes it into JSON format, and sends the data to an API endpoint that sends an HTTP response.

[0496] Step 7: Receiving and Viewing Messages

[0497] The device receives the HTTP response from the server, parses the received JSON data, and displays the parsed message in the user interface.

[0498] Input: JSON formatted HTTP response sent from the server

[0499] Output: The message that appears in the user interface

[0500] Specific behavior: The device listens for HTTP responses, parses the received data in JSON format, and displays the results in the application's UI.

[0501] These steps allow users to easily input their complaints and worries and receive instant feedback, thereby reducing mental stress and providing psychological support.

[0502] (Application example 1)

[0503] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0504] In recent years, there has been a demand for improved customer service and customer satisfaction in brick-and-mortar stores. However, due to a lack of appropriate systems for responding to minor concerns and complaints from customers, customer dissatisfaction can easily accumulate. It is also difficult to communicate effectively with customers and propose appropriate products and services.

[0505] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0506] In this invention, the server includes means for receiving complaints and worries input by a user using a mobile information terminal, natural language processing means for analyzing the received input data and extracting emotions and keywords, means for generating empathy and counseling messages based on the analyzed data, means for displaying the generated messages, means for equipping a robot with a touch screen and communicating with customers in the store, and means for proposing products and services of the store based on the customer's input. This makes it possible to provide immediate feedback on customers' worries and complaints and to propose products and services that meet the customer's needs.

[0507] A "personal digital assistant" is an electronic device that a user can carry and that allows the user to input and transmit data such as text and voice.

[0508] "Input data" refers to complaints, worries, and other text information entered by a user using a mobile information terminal or the touch screen of a robot.

[0509] "Natural language processing" refers to the technology and process for analyzing input data and extracting emotions and keywords from it.

[0510] "Empathy and counseling message" refers to a message containing empathy and advice for the user's situation, which is generated based on the analysis of data entered by the user.

[0511] "Display means" refers to techniques and devices for visually presenting the generated message to the user.

[0512] "Equipping a robot with a touchscreen" refers to attaching a touch-operable display to the exterior of the robot.

[0513] "Means of communicating with customers in-store" refers to the robot physically roaming the store and interacting with customers through touchscreens and voice.

[0514] "Means for suggesting products and services" refers to the technology and process that provides information about appropriate products and services based on customer input.

[0515] The present invention is a system for improving customer service in brick-and-mortar stores by analyzing complaints and worries entered by users using terminals, generating empathy and counseling messages, and even suggesting appropriate products and services. Specific embodiments of the present invention will be described below.

[0516] User Input

[0517] Users can use the touchscreen of the in-store robot to input their complaints and worries, for example, "I'm so busy at work I don't have time to rest." The input data is then captured by the robot.

[0518] Sending data

[0519] The text data entered by the user is sent from the robot to the server using an HTTP POST request.

[0520] Data analysis

[0521] The server analyzes the text data received from the device using a Natural Language Processing (NLP) engine (e.g., TextBlob). Through analysis, emotions and keywords from the input text are extracted. For example, for an input such as "I'm so busy at work I don't have time to rest," the emotion extracted is "stress," and the keywords extracted are "work," "busy," and "time to rest."

[0522] Creating a response message

[0523] The server generates empathy and counseling messages based on the analysis results. For example, if the analysis results indicate "stress," it generates a message such as, "That's really tough. It's important to take a break." This message shows specific empathy for the user's situation and provides practical advice.

[0524] Sending and displaying the response

[0525] The generated message is formatted in JSON format and sent back to the robot. The robot receives this message and displays it to the user. By seeing the displayed message, the user can feel that their situation is understood, which may reduce their mental burden.

[0526] Product and service suggestions

[0527] Furthermore, the system can also suggest products and services from stores related to the complaints and worries received. For example, if a user inputs that they are "too busy at work and don't have time to rest," the system can suggest products and services that have a relaxing effect.

[0528] Specific examples

[0529] Prompt Sentence Example 1:

[0530] Enter: "I'm so busy at work I don't have time to rest."

[0531] Response: "That's really tough. It's important to get some rest."

[0532] Prompt Sentence Example 2:

[0533] Enter: "I've been feeling down lately."

[0534] Response: "I get that sometimes. I'm happy to help."

[0535] The server analyzes the data entered by the user, generates optimal messages, and suggests related products and services. This system can increase customer satisfaction and improve the quality of customer service in stores.

[0536] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0537] Step 1:

[0538] The user inputs their complaints and worries into the robot's touchscreen. The user inputs the text "I'm too busy at work and I don't have time to rest" into the robot's touchscreen. The input data is captured by the terminal.

[0539] Step 2:

[0540] The device sends the user's input data to the server using an HTTP POST request. The device converts the input text data into JSON format and sends it to the server.

[0541] Step 3:

[0542] The server receives the HTTP POST request and analyzes the input data. The server uses a Natural Language Processing (NLP) engine (e.g., TextBlob) to extract emotions and keywords from the input text. The input is the text data sent by the user, and the output is the emotion "stress" and keywords such as "work," "busy," and "time to rest."

[0543] Step 4:

[0544] The server generates empathy and counseling messages based on the analysis results. Using a generative AI model, it creates a message for the emotion "stress" such as "That's really tough. It's important to take a break." The input is the analysis results, and the output is the generated empathy and counseling message.

[0545] Step 5:

[0546] The server formats the generated message in JSON format and sends it back to the device. The input is the empathy and counseling message, and the output is the message formatted in JSON format.

[0547] Step 6:

[0548] The device displays the message received from the server. Specifically, the user can see the message "This is really hard work. It's important to take a break." on the robot's touchscreen.

[0549] Step 7:

[0550] The server proposes products and services from the store that are related to the customer's complaints and worries. For example, it provides information on products (such as relaxation goods) and services (such as relaxation services) related to "stress" and "fatigue." The input is emotions and related keywords, and the output is related product proposal information.

[0551] Step 8:

[0552] The terminal displays suggested products and services to the user, who can check suggestions such as "relaxation goods" and "relaxation services" on the robot's touchscreen.

[0553] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0554] The present invention is a system that receives complaints and worries input by a user using a mobile information terminal, and a server analyzes the input and generates an appropriate message. Furthermore, by combining it with an emotion engine, the system can more accurately determine the user's emotional state and provide individual empathy and counseling messages. Specific embodiments for implementing the present invention are described below.

[0555] User Input

[0556] A user launches an app using a mobile information device. The app displays a prompt, "Hello. What's up?" and provides a text input field. The user enters their complaints or worries into the text input field. For example, they might enter, "I'm so busy at work, I don't have time to rest." Then, by pressing the "Send" button, the entered data is retrieved by the device.

[0557] Sending data

[0558] After receiving the user's input, the terminal sends the text data to the server using an HTTP POST request and waits for a response from the server.

[0559] Data analysis

[0560] The server analyzes the text data received from the device. For analysis, it uses a natural language processing engine and an emotion engine to extract emotions and keywords from the input text. For example, for an input such as "I'm so busy at work I don't have time to rest," the natural language processing engine extracts "stress" as the emotion and "work," "busy," and "time to rest" as keywords.

[0561] Emotion recognition

[0562] The server uses an emotion engine to recognize the user's specific emotional state based on the extracted keywords and context, for example, determining the level of "stress" and other related emotions (e.g., "anxiety" or "fatigue"). This allows for a more precise understanding of the user's emotional state.

[0563] Creating a response message

[0564] The server generates empathy and counseling messages based on the analysis results and the recognized emotional state. For example, if it determines that the user is experiencing "stress," it generates a message like, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you." Furthermore, if the emotion engine determines that "anxiety" or "fatigue" are also present, it generates additional messages accordingly.

[0565] Sending and displaying the response

[0566] The server formats the generated message into JSON format and sends it back to the device. The device receives this response and displays it to the user. By seeing the displayed message, the user feels that their situation is understood, which may reduce their mental burden.

[0567] For example, a user who types "I'm too busy at work and don't have time to rest" may receive feedback like this:

[0568] AI: "That's really tough. I hope you can find some time to rest, even if just a little. If you need it, don't push yourself too hard and ask for help. I'm also concerned about your anxiety. I understand that and we can try to find some solutions, little by little."

[0569] This system allows users to receive psychological support relatively easily. It is also convenient because users can input their complaints and worries at any time, regardless of location or time, and receive instant feedback. The addition of an emotion engine can further enhance this effective method of maintaining mental health by recognizing the user's emotional state in more detail and providing corresponding messages.

[0570] The processing flow will be explained below.

[0571] Step 1:

[0572] A user launches an app using a mobile device. The app displays a prompt saying "Hello, how can I help you?" and provides a text input field.

[0573] Step 2:

[0574] The user enters their complaints or worries into the text input field, for example, "I'm so busy at work I don't have time to rest," and then clicks the "Send" button.

[0575] Step 3:

[0576] The device obtains user input and sends the obtained text data to the server as an HTTP POST request to a pre-configured API endpoint.

[0577] Step 4:

[0578] The server receives the HTTP request and retrieves the text data, which is then passed to a natural language processing engine to extract sentiment and keywords.

[0579] Step 5:

[0580] The server uses a natural language processing engine to analyze the text data. For example, for an input such as "I'm so busy at work I don't have time to rest," it extracts "stress" as the emotion and "work," "busy," and "time to rest" as keywords.

[0581] Step 6:

[0582] The server uses an emotion engine to recognize the user's specific emotional state based on the extracted keywords and context, for example, determining the degree of "stress" and other related emotions (e.g., "anxiety" or "fatigue").

[0583] Step 7:

[0584] The server generates empathy and counseling messages based on the analysis results and the recognition results of the emotion engine. For example, if it determines that the symptom is "stress" and "anxiety" is also present, it generates a message saying, "That must be really tough. It would be good if you could find some time to rest. If necessary, try not to push yourself and ask for help from those around you. I'm also concerned about your anxiety. Let's understand that and try to think of ways to deal with it little by little."

[0585] Step 8:

[0586] The server formats the generated message into JSON format and sends the formatted data to the terminal as an HTTP response.

[0587] Step 9:

[0588] The device receives the response from the server, parses the received JSON data, and extracts the message.

[0589] Step 10:

[0590] The device displays the extracted message. The user confirms the displayed message. For example, a message might appear that reads, "AI: That must be really tough. I hope you can find some time to rest, even if just a little. If necessary, don't push yourself too hard and ask for help from those around you. I'm also concerned about your anxiety. I understand that, and let's try to come up with solutions little by little."

[0591] The above steps will realize a system that allows users to receive instant empathy and counseling for their complaints and worries. The addition of an emotion engine will enable a deeper understanding of the user's emotional state and provide personalized feedback accordingly.

[0592] Example 2

[0593] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0594] In modern society, mental strain caused by stress, anxiety, and constant busy schedules is a common problem for many people. However, in our busy daily lives, opportunities to appropriately share these concerns and receive professional feedback are limited. Furthermore, conventional systems have had difficulty accurately recognizing a user's emotions and providing individually tailored feedback. The present invention aims to provide a system that analyzes complaints and worries entered by a user, accurately grasps their emotional state, and provides empathy and counseling messages.

[0595] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for analyzing received input data using a natural language processing engine and an emotion engine and extracting emotions and keywords, a means for generating empathy and counseling messages based on the analyzed data, and a means for formatting the generated messages in JSON format and transmitting them to the mobile information terminal. This makes it possible to grasp the user's emotional state in more detail and provide appropriate feedback according to the individual situation.

[0596] A "personal digital assistant" is a mobile device used by a user, such as a smartphone or tablet, that is capable of text input and communication.

[0597] "Complaints and worries" are matters related to dissatisfaction, worries, stress, and difficulties that users feel in their daily lives and at work.

[0598] A "server" is a computer system that receives, analyzes, and processes data sent by users.

[0599] "Means for receiving" refers to the process or technology for transferring data from a mobile information terminal to a server, and includes, for example, communication technology using an HTTP POST request.

[0600] A "natural language processing engine" is software for analyzing text data and extracting grammatical and semantic information, and has functions such as morphological analysis and keyword extraction.

[0601] An "emotion engine" is software that analyzes emotional states from text data and recognizes the classification and intensity of emotions.

[0602] "Means for analyzing" refers to the process of processing received data using a natural language processing engine and an emotion engine to extract necessary information.

[0603] The "empathy and counseling message" is a feedback message generated according to the user's emotions and circumstances, and includes content that sympathizes with the user's feelings and provides appropriate advice and comfort.

[0604] "Generative means" refers to the processes and techniques for constructing empathy and counseling messages based on the analyzed data.

[0605] The "JSON format" is a lightweight data format for exchanging data that is easy to read for both humans and machines.

[0606] "Means for sending" refers to the process for delivering the generated message to the mobile information terminal, and includes, for example, data communication via the Internet.

[0607] "Displaying means" refers to the process or technique for visually presenting a message to a user on a mobile device.

[0608] MODE FOR CARRYING OUT THE INVENTION

[0609] This system receives complaints and worries entered by users using a mobile information terminal, and the server analyzes them to generate appropriate messages. Furthermore, by combining it with an emotion engine, the system can more accurately determine the user's emotional state and provide personalized empathy and counseling messages.

[0610] Specific Embodiments

[0611] User Input

[0612] A user launches an app using a mobile information device. The app displays a prompt, "Hello. What's up?" and provides a text input field. The user enters their complaints or worries into the text input field. For example, they might enter, "I'm so busy at work I don't have time to rest." Then, by pressing the "Send" button, the entered data is retrieved by the device.

[0613] Sending data

[0614] After receiving the user's input, the device sends the text data to the server using an HTTP POST request and waits for a response from the server. The destination URL is set in advance.

[0615] Data analysis

[0616] The server analyzes the text data received from the device. For analysis, it uses a Python-based natural language processing engine (e.g., spaCy) and an emotion engine (e.g., Hugging Face Transformers). It performs morphological analysis of the input text and extracts keywords. For example, for the input "I'm so busy at work I don't have time to rest," the natural language processing engine extracts "stress" as the emotion and "work," "busy," and "time to rest" as keywords.

[0617] Emotion recognition

[0618] The server uses an emotion engine to recognize the user's specific emotional state based on the extracted keywords and context. The emotion engine uses the BERT model to determine the level of "stress" and other related emotions (e.g., "anxiety" or "fatigue"). This process allows for a more precise understanding of the user's emotional state.

[0619] Creating a response message

[0620] The server generates empathy and counseling messages based on the analysis results and the recognized emotional state. The program uses conditional branching to select or generate appropriate messages. For example, if the condition is determined to be "stress," it generates a message saying, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help." If the emotion engine also determines that "anxiety" or "fatigue" are present, it generates additional messages accordingly.

[0621] Sending and displaying the response

[0622] The server formats the generated message into JSON format and sends it back to the device. The device receives this response and displays it to the user. The display is in text format on the device screen, and by reading the message, the user feels that their situation is understood, which may reduce their mental burden.

[0623] Examples and prompts

[0624] Below is a specific example and the prompt to input to the generative AI model.

[0625] example

[0626] User input: "I'm too busy at work and I don't have time to rest."

[0627] Generated feedback:

[0628] That must be really tough. I hope you can find some time to rest, even if just a little. If you need it, don't push yourself too hard and ask for help from those around you. I'm also concerned about your anxiety. Understand that and try to think of ways to deal with it little by little.

[0629] Prompt Sentence Examples

[0630] User input: "I'm too busy at work and I don't have time to rest."

[0631] Generative AI model input prompt:

[0632] User input: "I'm too busy at work and I don't have time to rest."

[0633] System:

[0634] The natural language processing engine analyzes the data and extracts "stress," "work," "busy," and "rest time." The emotion engine recognizes specific emotional states (e.g., stress, anxiety, fatigue) and generates appropriate messages.

[0635] This system allows users to receive psychological support relatively easily. It is also convenient because users can input their complaints and worries at any time, regardless of location or time, and receive instant feedback. The addition of an emotion engine can further enhance this effective method of maintaining mental health by recognizing the user's emotional state in more detail and providing corresponding messages.

[0636] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0637] Step 1:

[0638] A user launches an app using a mobile information device. The app displays a prompt saying "Hello. What's up?" and provides a text input field. The user enters their complaints or worries into the text input field and presses the "Send" button. Specifically, the device acquires the user's text input and prepares it for transmission. The input includes the text of the user's worries or complaints. The output is the acquired text data.

[0639] Step 2:

[0640] The device sends the user's input to the server as an HTTP POST request. This request contains the text data entered by the user. Specifically, the device sends data to the destination URL and waits for a response from the server. The input is the text data entered by the user, and the output is data sent to the server.

[0641] Step 3:

[0642] The server analyzes the text data received from the terminal. First, a natural language processing engine (e.g., spaCy) is used to perform morphological analysis and extract keywords. The text data is input for this analysis. Specifically, through morphological analysis, the keywords "work," "busy," and "time to rest" are extracted. The extracted keywords are generated as output.

[0643] Step 4:

[0644] The server uses an emotion engine (e.g., Hugging Face Transformers) to analyze the emotional state of the text data. The input is the keywords extracted in the previous step. Specifically, the emotion engine analyzes the context of the text and recognizes emotions such as "stress" and "anxiety." The output is the recognized emotional state.

[0645] Step 5:

[0646] The server generates empathy and counseling messages based on the analysis results and the recognized emotional state. The inputs are the recognized emotional state and extracted keywords. Specific operations include using conditional branching to select an appropriate message. For example, if it is determined to be "stressed," it generates a message saying, "That must be really tough. It would be good if you could find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you." The generated message is output.

[0647] Step 6:

[0648] The server formats the generated message into JSON format and sends it back to the terminal. The input is the generated message. Specifically, the generated message is converted into JSON format and an HTTP POST request is sent to the terminal again. The formatted message is generated as the output.

[0649] Step 7:

[0650] The device displays the received message to the user. The input is the received JSON format message. Specifically, the device parses the message and displays it on the screen. By reading this, the user feels that their feelings have been understood. The displayed message is generated as the output.

[0651] Through these steps, users can receive psychological support relatively easily. This system accurately grasps the user's emotional state and provides appropriate feedback to maintain mental health. It is also available at any time, regardless of location or time.

[0652] (Application example 2)

[0653] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0654] In recent years, technologies for improving consumer purchasing experiences have been attracting attention. In particular, brick-and-mortar stores are required to quickly resolve customer problems and provide appropriate support. However, current systems have difficulty accurately understanding customers' emotions and concerns and providing appropriate empathy and counseling messages. Furthermore, when using mobile information terminals to achieve this, real-time feedback and support are insufficient. This invention aims to solve these problems and quickly resolve problems customers face in brick-and-mortar stores.

[0655] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving complaints and worries input by a user using a mobile information terminal, natural language processing means for analyzing the received input data and extracting emotions and keywords, means for generating empathy and counseling messages based on the analyzed data, and means for displaying the generated messages on the mobile information terminal or the wearable display device. This makes it possible to provide immediate and appropriate support for problems users encounter in physical stores and improve customer experience.

[0656] "Mobile information terminal" refers to an electronic device that can be carried by a user and that can input and display data. Specifically, it includes smartphones and tablet terminals.

[0657] "Wearable display devices" refer to display devices that can be worn by the user. Examples include smart glasses and head-mounted displays.

[0658] "Natural language processing means" refers to technology that analyzes text data entered by users and extracts emotions and keywords. Specifically, this is done using a natural language processing engine.

[0659] "Emotion analysis engine" refers to technology that analyzes emotions contained in received text data and determines the user's emotional state, enabling a more accurate understanding of the user's emotions.

[0660] An "empathetic message" is a message that shows empathy for the user's feelings and situation and understands how the user feels, thereby reducing the user's mental burden.

[0661] A "counseling message" is a message that provides specific advice or suggestions for the problems or worries that the user is having, thereby helping the user to solve the problem.

[0662] A "server" is a computer system that receives and processes data sent by users. It uses various engines to analyze data and generate messages.

[0663] This invention is a system that receives complaints and worries input by a user using a mobile information terminal or a wearable display device, analyzes the input, and generates appropriate empathy and counseling messages. Detailed embodiments of this system are described below.

[0664] User Input

[0665] A user launches an application using a mobile information terminal (such as a smartphone or tablet) or a wearable display device (such as smart glasses). The application displays a prompt, saying, "Please let us know if you have any problems or complaints about the store," and provides a voice or text input field. For example, the user might type, "The store is too crowded and I feel uneasy. I can't even find the restroom." Then, by pressing the "Submit" button, the input data is captured by the device.

[0666] Sending data

[0667] After the terminal receives the user's input, it sends the text data to the server using an HTTP POST request and waits for a response from the server. At this time, it uses the requests module to send the data to the server.

[0668] Data analysis

[0669] The server analyzes the text data received from the device. A natural language processing engine and a sentiment analysis engine are used for the analysis, and emotions and keywords from the input text are extracted. For example, in response to an input such as "The store is too crowded and I feel uneasy. I don't know where the restroom is," the natural language processing engine extracts "stress due to crowding" as the emotion and "inside the store," "crowded," and "restroom" as keywords.

[0670] Emotion recognition

[0671] The server uses an emotion analysis engine to recognize the user's specific emotional state based on the extracted keywords and context. For example, it determines the degree of "stress due to crowding." It also determines other related emotions (e.g., "anxiety" or "dissatisfaction"), allowing for a more precise understanding of the user's emotional state.

[0672] Creating a response message

[0673] The server generates empathy and counseling messages based on the analysis results and the recognized emotional state. For example, if it determines that the customer is feeling "stressed due to crowding," it generates a message saying, "It seems you're worried about the store being crowded. The restroom is located at the back right of the store. If you wait a moment, a staff member will assist you." If the emotion analysis engine also determines that "anxiety" is present, it generates an additional message accordingly.

[0674] Sending and displaying the response

[0675] The server formats the generated message into JSON format and sends it back to the device. The device receives this response and displays it to the user. Seeing the displayed message makes the user feel that their situation is understood, which may reduce their mental burden. For example, a user who types, "The store is too crowded and I feel uneasy. I can't even find the restroom," could receive the following feedback:

[0676] Prompt Sentence Examples

[0677] User input: "The store is too crowded and I feel uneasy. I don't know where the restroom is."

[0678] Feedback message: "I understand you're concerned about the store being busy. The restroom is located at the back right of the store. Please wait a moment and a member of staff will help you."

[0679] The system allows users to receive quick support for any issues they may encounter in-store, improving their shopping experience.

[0680] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0681] Step 1:

[0682] A user launches an application using a mobile information terminal or a wearable display device. The terminal displays a prompt on the screen saying, "Please let us know if you have any problems or complaints about the store," and provides voice and text input fields. The input here is the user's complaints or worries. For example, the user might enter, "The store is too crowded and I feel uneasy. I can't even find the restroom."

[0683] input:

[0684] Text of the user's complaints or worries (e.g., "The store is too crowded and I can't relax. I don't know where the restroom is.")

[0685] output:

[0686] User input text data

[0687] Step 2:

[0688] The terminal obtains the user's input text data and sends it to the server using an HTTP POST request, using the requests module to send the data to the server.

[0689] input:

[0690] User input text data

[0691] output:

[0692] Text data sent to the server

[0693] Step 3:

[0694] The server analyzes the received text data. Using a natural language processing engine, the server extracts emotions and keywords from the input text. In this example, for the text "The store is too crowded and I feel uneasy. I don't even know where the restroom is," the server extracts "stress due to crowding" as the emotion, and extracts "inside the store," "crowded," and "restroom" as keywords.

[0695] input:

[0696] Text data sent to the server

[0697] output:

[0698] Extracted emotions and keywords

[0699] Step 4:

[0700] The server uses an emotion analysis engine to determine the user's specific emotional state based on the extracted keywords and context. For example, it evaluates the degree of "stress due to crowding" and determines related emotions. At this time, it also analyzes whether other emotions (such as "anxiety" or "dissatisfaction") are present.

[0701] input:

[0702] Extracted emotions and keywords

[0703] output:

[0704] The user's specific emotional state

[0705] Step 5:

[0706] The server generates empathy and counseling messages based on the analysis results and the recognized emotional state. For example, if it determines that the customer is feeling "stressed due to crowding," it generates a message saying, "It seems you're worried about the store being crowded. The restroom is located at the back right of the store. If you wait a moment, a staff member will assist you." If the emotion analysis engine determines that "anxiety" is also present, it generates an additional message accordingly.

[0707] input:

[0708] The user's specific emotional state

[0709] output:

[0710] Generated empathy and counseling messages

[0711] Step 6:

[0712] The server formats the generated message into JSON format and sends it back to the device. The device receives this response and displays it to the user. Seeing the displayed message can help the user feel that their situation is understood, which can reduce their mental burden.

[0713] input:

[0714] Generated empathy and counseling messages

[0715] output:

[0716] Feedback message displayed on the device

[0717] This process allows users to receive prompt and appropriate support for any issues they may have, improving their shopping experience in physical stores.

[0718] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0719] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0720] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0721] [Third embodiment]

[0722] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0723] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0724] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0725] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0726] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0727] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0728] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0729] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0730] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0731] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0732] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0733] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0734] The present invention is a system in which a user inputs complaints or worries using a mobile information terminal, and a server analyzes the input and generates an appropriate message. Specific embodiments for carrying out the present invention will be described below.

[0735] User Input

[0736] A user starts an application using a mobile information terminal and inputs their complaints or worries. For example, they can type something like "I'm so busy at work I don't have time to rest" directly into the text input field. Then, by pressing the "Send" button, the input data is retrieved by the terminal.

[0737] Sending data

[0738] After receiving the user's input, the terminal sends the text data to the server via an HTTP POST request. The terminal sends this request and waits for a response from the server.

[0739] Data analysis

[0740] The server analyzes the text data received from the device. A natural language processing engine is used for the analysis to extract emotions and keywords from the input text. For example, for an input such as "I'm so busy at work I don't have time to rest," the analysis engine extracts "stress" as the emotion and "work," "busy," and "time to rest" as keywords.

[0741] Creating a response message

[0742] The server generates empathy and counseling messages based on the analysis results. For example, if the analysis results indicate "stress," it generates a message such as, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you." This message provides specific empathy and practical advice for the user's situation.

[0743] Sending and displaying the response

[0744] The server formats the generated message into JSON format and sends it back to the device. The device receives this response and displays it to the user. By seeing the displayed message, the user feels that their situation is understood, which may reduce their mental burden.

[0745] For example, a user who types "I'm too busy at work and don't have time to rest" may receive feedback like this:

[0746] AI: "That's really tough. I hope you can find some time to rest, even if it's just for a little while. Don't push yourself too hard, and ask for help if you need it."

[0747] This system allows users to receive psychological support relatively easily. It is also convenient because users can input their complaints and worries at any time, regardless of location or time, and receive instant feedback. This makes it possible to provide an effective means of reducing individual stress and worries and maintaining mental health.

[0748] The processing flow will be explained below.

[0749] Step 1:

[0750] A user launches an app using a mobile device. The app displays a prompt saying "Hello, how can I help you?" and provides a text input field.

[0751] Step 2:

[0752] The user enters their complaint or worry into the text input field, for example, "I'm so busy at work I don't have time to rest." Then, the user clicks the "Send" button.

[0753] Step 3:

[0754] The device obtains user input and sends the obtained text data to the server as an HTTP POST request to a pre-configured API endpoint.

[0755] Step 4:

[0756] The server receives the HTTP request, retrieves the text data, and passes it to a natural language processing engine for analysis.

[0757] Step 5:

[0758] The server uses a natural language processing engine to analyze the text data and extract emotions and keywords. For example, for an input such as "I'm so busy at work I don't have time to rest," the server extracts "stress" as the emotion and "work," "busy," and "time to rest" as keywords.

[0759] Step 6:

[0760] The server generates empathy and counseling messages based on the analysis results. For example, if the server determines that the patient is suffering from stress, it generates a message such as, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you."

[0761] Step 7:

[0762] The server formats the generated message into JSON format and sends the formatted data to the terminal as an HTTP response.

[0763] Step 8:

[0764] The device receives the response from the server, parses the received JSON data, and extracts the message.

[0765] Step 9:

[0766] The device displays the extracted message. The user confirms the displayed message. For example, a message like "AI: That's really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you" may be displayed.

[0767] The above steps realize a system that allows users to instantly receive sympathy and counseling for their complaints and worries.

[0768] Example 1

[0769] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0770] In modern society, many people suffer from stress and worries in their work and personal lives. The lack of an environment where people can easily seek advice about these worries leads to an accumulation of mental stress, creating a problem. Furthermore, time and financial constraints mean that many people are unable to receive appropriate support from professional counseling. For this reason, there is a need for a system that allows users to easily input their complaints and worries and receive feedback on them.

[0771] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0772] In this invention, the server includes means for receiving text data from a mobile information terminal, means for transmitting the received text data as an HTTP POST request, means including a natural language processing engine for analyzing the received text data, means for generating empathy and counseling messages based on the analysis results, and means for formatting the generated messages into JSON format and transmitting them again to the mobile information terminal. This allows users to easily input their complaints and worries and receive instant feedback, thereby reducing mental stress and providing psychological support.

[0773] "User" refers to an individual who uses the system to input complaints and worries and receive feedback.

[0774] "Mobile information terminal" refers to a portable computer terminal that can connect to the Internet, such as a smartphone or tablet.

[0775] "Text data" refers to text information input by a user using a mobile information terminal.

[0776] An "HTTP POST request" is one of the HTTP protocol methods for a client to send data to a server.

[0777] "Server" refers to a computer system that receives requests from clients over a network and processes and responds to the requests.

[0778] A "natural language processing engine" refers to a software system that analyzes input text data and extracts emotions and keywords.

[0779] "Emotion" refers to the user's psychological state analyzed from text data.

[0780] "Keywords" refer to important words or phrases in the text data.

[0781] A "sympathetic message" refers to a sentence that shows understanding and sympathy for the user's input.

[0782] A "counseling message" refers to a piece of text that provides advice and support for a user's concerns.

[0783] "JSON format" is an abbreviation for JavaScript Object Notation and refers to a lightweight data format for data exchange.

[0784] The present invention is a system in which a user inputs complaints or worries using a mobile information terminal, and a server analyzes the input and generates an appropriate message. Specifically, the present invention is implemented using the following hardware and software.

[0785] Provided environment

[0786] Hardware and Software

[0787] 1. Personal digital assistant: A mobile device such as a smartphone or tablet, which includes an input / output interface (e.g., a touchscreen) and internet connectivity.

[0788] 2. Server: A powerful computer system that communicates with clients over the Internet and analyzes the data it receives.

[0789] 3. Natural Language Processing Engine: A software system used to analyze incoming text data, such as the Google Cloud Natural Language API, to extract sentiment and keywords from the text entered by the user.

[0790] 4. API Requests and Responses: HTTP POST requests and responses are used as the communication protocol for exchanging data between the mobile information terminal and the server.

[0791] Implementation method

[0792] 1. User Input

[0793] The user uses a mobile information terminal to launch a dedicated application and enter their complaints or worries into the text field. For example, they can type, "I'm so busy at work I don't have time to rest," and press the "Send" button. This operation causes the terminal to acquire the user's input data.

[0794] 2. Data transmission

[0795] The device receives the text data entered by the user and sends it to the server as an HTTP POST request, along with metadata such as the user ID and a timestamp, so the server is ready to process the data.

[0796] 3. Receiving and analyzing text data

[0797] The server receives a POST request sent from the device. The received data is decoded and passed to a natural language processing engine for sentiment analysis and keyword extraction. For example, from the text "I'm so busy at work I don't have time to rest," the emotion "stress" and the keywords "work," "busy," and "time to rest" are extracted.

[0798] 4. Creating a Response Message

[0799] The server generates empathy and counseling messages based on the analysis results. For example, if it determines that the user is experiencing stress, it generates a message such as, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you."

[0800] 5. Sending and Displaying the Response

[0801] The server formats the generated message into JSON format and sends it back to the device as an HTTP response. The device receives the response from the server, parses the received data, and displays it in the user interface. This allows the user to receive feedback and feel that their situation is understood.

[0802] Examples and prompts

[0803] If the user types "I'm too busy at work and don't have time to rest," the following feedback will be displayed:

[0804] Specific examples

[0805] User input:

[0806] "I'm so busy with work that I don't have time to rest"

[0807] Server Feedback:

[0808] "That must be really hard. I hope you can find some time to rest, even if just a little. If you need it, don't push yourself too hard and ask for help from those around you."

[0809] Prompt Sentence Examples

[0810] User input: "I'm too busy at work and I don't have time to rest."

[0811] Message to generate: A message that contains empathy and specific advice to reduce the user's stress.

[0812] In this way, we provide a system that allows users to easily input their complaints and worries on a mobile information terminal and receive appropriate feedback immediately, thereby reducing the user's mental burden and providing psychological support.

[0813] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0814] Step 1: User Input

[0815] A user starts the application using a mobile information terminal, enters their complaints or worries into the text input field, for example, "I'm so busy at work I don't have time to rest," and presses the "Send" button.

[0816] Input: User text input

[0817] Output: Text data acquired within the device when the send button is pressed

[0818] Specific behavior: The user taps the app to launch it, enters text in the text field, and presses the send button.

[0819] Step 2: Send data

[0820] The device receives the text data and sends it to the server as an HTTP POST request, which also includes metadata such as the user ID and a timestamp.

[0821] Input: Text data and metadata acquired within the device

[0822] Output: Data sent to the server as an HTTP POST request

[0823] Specific operation: The terminal stores the input data in a buffer, generates an HTTP POST request, and sends it to the server.

[0824] Step 3: Receiving text data

[0825] The server receives the HTTP POST request sent from the terminal and decodes it to extract text data and metadata.

[0826] Input: HTTP POST request from terminal

[0827] Output: Extracted text data and metadata

[0828] What it does: The server listens for HTTP requests and parses the data to get the required information.

[0829] Step 4: Analyzing the text data

[0830] The server uses a natural language processing engine to analyze the received text data and extract emotions and keywords. For example, in the case of "I'm so busy at work I don't have time to rest," the server extracts the emotion "stress" and the keywords "work," "busy," and "time to rest."

[0831] Input: Extracted text data

[0832] Output: Parsed sentiment and keywords

[0833] What happens: The server sends an API request to the natural language processing engine and runs a script to parse the results.

[0834] Step 5: Generate a response message

[0835] The server generates empathy and counseling messages based on the analysis results. For example, if the server determines that the patient is suffering from stress, it generates a message such as, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you."

[0836] Input: Parsed sentiment and keywords

[0837] Output: Generated empathy and counseling messages

[0838] Specific operation: The server executes a script that generates a message using a predefined message template based on the analysis results.

[0839] Step 6: Sending the Response

[0840] The server formats the generated message into JSON format and resends it to the terminal as an HTTP response.

[0841] Input: Generated empathy and counseling messages

[0842] Output: HTTP response in structured JSON data format

[0843] Specific operation: The server generates a message, encodes it into JSON format, and sends the data to an API endpoint that sends an HTTP response.

[0844] Step 7: Receiving and Viewing Messages

[0845] The device receives the HTTP response from the server, parses the received JSON data, and displays the parsed message in the user interface.

[0846] Input: JSON formatted HTTP response sent from the server

[0847] Output: The message that appears in the user interface

[0848] Specific behavior: The device listens for HTTP responses, parses the received data in JSON format, and displays the results in the application's UI.

[0849] These steps allow users to easily input their complaints and worries and receive instant feedback, thereby reducing mental stress and providing psychological support.

[0850] (Application example 1)

[0851] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0852] In recent years, there has been a demand for improved customer service and customer satisfaction in brick-and-mortar stores. However, due to a lack of appropriate systems for responding to minor concerns and complaints from customers, customer dissatisfaction can easily accumulate. It is also difficult to communicate effectively with customers and propose appropriate products and services.

[0853] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0854] In this invention, the server includes means for receiving complaints and worries input by a user using a mobile information terminal, natural language processing means for analyzing the received input data and extracting emotions and keywords, means for generating empathy and counseling messages based on the analyzed data, means for displaying the generated messages, means for equipping a robot with a touch screen and communicating with customers in the store, and means for proposing products and services of the store based on the customer's input. This makes it possible to provide immediate feedback on customers' worries and complaints and to propose products and services that meet the customer's needs.

[0855] A "personal digital assistant" is an electronic device that a user can carry and that allows the user to input and transmit data such as text and voice.

[0856] "Input data" refers to complaints, worries, and other text information entered by a user using a mobile information terminal or the touch screen of a robot.

[0857] "Natural language processing" refers to the technology and process for analyzing input data and extracting emotions and keywords from it.

[0858] "Empathy and counseling message" refers to a message containing empathy and advice for the user's situation, which is generated based on the analysis of data entered by the user.

[0859] "Display means" refers to techniques and devices for visually presenting the generated message to the user.

[0860] "Equipping a robot with a touchscreen" refers to attaching a touch-operable display to the exterior of the robot.

[0861] "Means of communicating with customers in-store" refers to the robot physically roaming the store and interacting with customers through touchscreens and voice.

[0862] "Means for suggesting products and services" refers to the technology and process that provides information about appropriate products and services based on customer input.

[0863] The present invention is a system for improving customer service in brick-and-mortar stores by analyzing complaints and worries entered by users using terminals, generating empathy and counseling messages, and even suggesting appropriate products and services. Specific embodiments of the present invention will be described below.

[0864] User Input

[0865] Users can use the touchscreen of the in-store robot to input their complaints and worries, for example, "I'm so busy at work I don't have time to rest." The input data is then captured by the robot.

[0866] Sending data

[0867] The text data entered by the user is sent from the robot to the server using an HTTP POST request.

[0868] Data analysis

[0869] The server analyzes the text data received from the device using a Natural Language Processing (NLP) engine (e.g., TextBlob). Through analysis, emotions and keywords from the input text are extracted. For example, for an input such as "I'm so busy at work I don't have time to rest," the emotion extracted is "stress," and the keywords extracted are "work," "busy," and "time to rest."

[0870] Creating a response message

[0871] The server generates empathy and counseling messages based on the analysis results. For example, if the analysis results indicate "stress," it generates a message such as, "That's really tough. It's important to take a break." This message shows specific empathy for the user's situation and provides practical advice.

[0872] Sending and displaying the response

[0873] The generated message is formatted in JSON format and sent back to the robot. The robot receives this message and displays it to the user. By seeing the displayed message, the user can feel that their situation is understood, which may reduce their mental burden.

[0874] Product and service suggestions

[0875] Furthermore, the system can also suggest products and services from stores related to the complaints and worries received. For example, if a user inputs that they are "too busy at work and don't have time to rest," the system can suggest products and services that have a relaxing effect.

[0876] Specific examples

[0877] Prompt Sentence Example 1:

[0878] Enter: "I'm so busy at work I don't have time to rest."

[0879] Response: "That's really tough. It's important to get some rest."

[0880] Prompt Sentence Example 2:

[0881] Enter: "I've been feeling down lately."

[0882] Response: "I get that sometimes. I'm happy to help."

[0883] The server analyzes the data entered by the user, generates optimal messages, and suggests related products and services. This system can increase customer satisfaction and improve the quality of customer service in stores.

[0884] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0885] Step 1:

[0886] The user inputs their complaints and worries into the robot's touchscreen. The user inputs the text "I'm too busy at work and I don't have time to rest" into the robot's touchscreen. The input data is captured by the terminal.

[0887] Step 2:

[0888] The device sends the user's input data to the server using an HTTP POST request. The device converts the input text data into JSON format and sends it to the server.

[0889] Step 3:

[0890] The server receives the HTTP POST request and analyzes the input data. The server uses a Natural Language Processing (NLP) engine (e.g., TextBlob) to extract emotions and keywords from the input text. The input is the text data sent by the user, and the output is the emotion "stress" and keywords such as "work," "busy," and "time to rest."

[0891] Step 4:

[0892] The server generates empathy and counseling messages based on the analysis results. Using a generative AI model, it creates a message for the emotion "stress" such as "That's really tough. It's important to take a break." The input is the analysis results, and the output is the generated empathy and counseling message.

[0893] Step 5:

[0894] The server formats the generated message in JSON format and sends it back to the device. The input is the empathy and counseling message, and the output is the message formatted in JSON format.

[0895] Step 6:

[0896] The device displays the message received from the server. Specifically, the user can see the message "This is really hard work. It's important to take a break." on the robot's touchscreen.

[0897] Step 7:

[0898] The server proposes products and services from the store that are related to the customer's complaints and worries. For example, it provides information on products (such as relaxation goods) and services (such as relaxation services) related to "stress" and "fatigue." The input is emotions and related keywords, and the output is related product proposal information.

[0899] Step 8:

[0900] The terminal displays suggested products and services to the user, who can check suggestions such as "relaxation goods" and "relaxation services" on the robot's touchscreen.

[0901] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0902] The present invention is a system that receives complaints and worries input by a user using a mobile information terminal, and a server analyzes the input and generates an appropriate message. Furthermore, by combining it with an emotion engine, the system can more accurately determine the user's emotional state and provide individual empathy and counseling messages. Specific embodiments for implementing the present invention are described below.

[0903] User Input

[0904] A user launches an app using a mobile information device. The app displays a prompt, "Hello. What's up?" and provides a text input field. The user enters their complaints or worries into the text input field. For example, they might enter, "I'm so busy at work, I don't have time to rest." Then, by pressing the "Send" button, the entered data is retrieved by the device.

[0905] Sending data

[0906] After receiving the user's input, the terminal sends the text data to the server using an HTTP POST request and waits for a response from the server.

[0907] Data analysis

[0908] The server analyzes the text data received from the device. For analysis, it uses a natural language processing engine and an emotion engine to extract emotions and keywords from the input text. For example, for an input such as "I'm so busy at work I don't have time to rest," the natural language processing engine extracts "stress" as the emotion and "work," "busy," and "time to rest" as keywords.

[0909] Emotion recognition

[0910] The server uses an emotion engine to recognize the user's specific emotional state based on the extracted keywords and context, for example, determining the level of "stress" and other related emotions (e.g., "anxiety" or "fatigue"). This allows for a more precise understanding of the user's emotional state.

[0911] Creating a response message

[0912] The server generates empathy and counseling messages based on the analysis results and the recognized emotional state. For example, if it determines that the user is experiencing "stress," it generates a message like, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you." Furthermore, if the emotion engine determines that "anxiety" or "fatigue" are also present, it generates additional messages accordingly.

[0913] Sending and displaying the response

[0914] The server formats the generated message into JSON format and sends it back to the device. The device receives this response and displays it to the user. By seeing the displayed message, the user feels that their situation is understood, which may reduce their mental burden.

[0915] For example, a user who types "I'm too busy at work and don't have time to rest" may receive feedback like this:

[0916] AI: "That's really tough. I hope you can find some time to rest, even if just a little. If you need it, don't push yourself too hard and ask for help. I'm also concerned about your anxiety. I understand that and we can try to find some solutions, little by little."

[0917] This system allows users to receive psychological support relatively easily. It is also convenient because users can input their complaints and worries at any time, regardless of location or time, and receive instant feedback. The addition of an emotion engine can further enhance this effective method of maintaining mental health by recognizing the user's emotional state in more detail and providing corresponding messages.

[0918] The processing flow will be explained below.

[0919] Step 1:

[0920] A user launches an app using a mobile device. The app displays a prompt saying "Hello, how can I help you?" and provides a text input field.

[0921] Step 2:

[0922] The user enters their complaints or worries into the text input field, for example, "I'm so busy at work I don't have time to rest," and then clicks the "Send" button.

[0923] Step 3:

[0924] The device obtains user input and sends the obtained text data to the server as an HTTP POST request to a pre-configured API endpoint.

[0925] Step 4:

[0926] The server receives the HTTP request and retrieves the text data, which is then passed to a natural language processing engine to extract sentiment and keywords.

[0927] Step 5:

[0928] The server uses a natural language processing engine to analyze the text data. For example, for an input such as "I'm so busy at work I don't have time to rest," it extracts "stress" as the emotion and "work," "busy," and "time to rest" as keywords.

[0929] Step 6:

[0930] The server uses an emotion engine to recognize the user's specific emotional state based on the extracted keywords and context, for example, determining the degree of "stress" and other related emotions (e.g., "anxiety" or "fatigue").

[0931] Step 7:

[0932] The server generates empathy and counseling messages based on the analysis results and the recognition results of the emotion engine. For example, if it determines that the symptom is "stress" and "anxiety" is also present, it generates a message saying, "That must be really tough. It would be good if you could find some time to rest. If necessary, try not to push yourself and ask for help from those around you. I'm also concerned about your anxiety. Let's understand that and try to think of ways to deal with it little by little."

[0933] Step 8:

[0934] The server formats the generated message into JSON format and sends the formatted data to the terminal as an HTTP response.

[0935] Step 9:

[0936] The device receives the response from the server, parses the received JSON data, and extracts the message.

[0937] Step 10:

[0938] The device displays the extracted message. The user confirms the displayed message. For example, a message might appear that reads, "AI: That must be really tough. I hope you can find some time to rest, even if just a little. If necessary, don't push yourself too hard and ask for help from those around you. I'm also concerned about your anxiety. I understand that, and let's try to come up with solutions little by little."

[0939] The above steps will realize a system that allows users to receive instant empathy and counseling for their complaints and worries. The addition of an emotion engine will enable a deeper understanding of the user's emotional state and provide personalized feedback accordingly.

[0940] Example 2

[0941] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0942] In modern society, mental strain caused by stress, anxiety, and constant busy schedules is a common problem for many people. However, in our busy daily lives, opportunities to appropriately share these concerns and receive professional feedback are limited. Furthermore, conventional systems have had difficulty accurately recognizing a user's emotions and providing individually tailored feedback. The present invention aims to provide a system that analyzes complaints and worries entered by a user, accurately grasps their emotional state, and provides empathy and counseling messages.

[0943] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for analyzing received input data using a natural language processing engine and an emotion engine and extracting emotions and keywords, a means for generating empathy and counseling messages based on the analyzed data, and a means for formatting the generated messages in JSON format and transmitting them to the mobile information terminal. This makes it possible to grasp the user's emotional state in more detail and provide appropriate feedback according to the individual situation.

[0944] A "personal digital assistant" is a mobile device used by a user, such as a smartphone or tablet, that is capable of text input and communication.

[0945] "Complaints and worries" are matters related to dissatisfaction, worries, stress, and difficulties that users feel in their daily lives and at work.

[0946] A "server" is a computer system that receives, analyzes, and processes data sent by users.

[0947] "Means for receiving" refers to the process or technology for transferring data from a mobile information terminal to a server, and includes, for example, communication technology using an HTTP POST request.

[0948] A "natural language processing engine" is software for analyzing text data and extracting grammatical and semantic information, and has functions such as morphological analysis and keyword extraction.

[0949] An "emotion engine" is software that analyzes emotional states from text data and recognizes the classification and intensity of emotions.

[0950] "Means for analyzing" refers to the process of processing received data using a natural language processing engine and an emotion engine to extract necessary information.

[0951] The "empathy and counseling message" is a feedback message generated according to the user's emotions and circumstances, and includes content that sympathizes with the user's feelings and provides appropriate advice and comfort.

[0952] "Generative means" refers to the processes and techniques for constructing empathy and counseling messages based on the analyzed data.

[0953] The "JSON format" is a lightweight data format for exchanging data that is easy to read for both humans and machines.

[0954] "Means for sending" refers to the process for delivering the generated message to the mobile information terminal, and includes, for example, data communication via the Internet.

[0955] "Displaying means" refers to the process or technique for visually presenting a message to a user on a mobile device.

[0956] MODE FOR CARRYING OUT THE INVENTION

[0957] This system receives complaints and worries entered by users using a mobile information terminal, and the server analyzes them to generate appropriate messages. Furthermore, by combining it with an emotion engine, the system can more accurately determine the user's emotional state and provide personalized empathy and counseling messages.

[0958] Specific Embodiments

[0959] User Input

[0960] A user launches an app using a mobile information device. The app displays a prompt, "Hello. What's up?" and provides a text input field. The user enters their complaints or worries into the text input field. For example, they might enter, "I'm so busy at work I don't have time to rest." Then, by pressing the "Send" button, the entered data is retrieved by the device.

[0961] Sending data

[0962] After receiving the user's input, the device sends the text data to the server using an HTTP POST request and waits for a response from the server. The destination URL is set in advance.

[0963] Data analysis

[0964] The server analyzes the text data received from the device. For analysis, it uses a Python-based natural language processing engine (e.g., spaCy) and an emotion engine (e.g., Hugging Face Transformers). It performs morphological analysis of the input text and extracts keywords. For example, for the input "I'm so busy at work I don't have time to rest," the natural language processing engine extracts "stress" as the emotion and "work," "busy," and "time to rest" as keywords.

[0965] Emotion recognition

[0966] The server uses an emotion engine to recognize the user's specific emotional state based on the extracted keywords and context. The emotion engine uses the BERT model to determine the level of "stress" and other related emotions (e.g., "anxiety" or "fatigue"). This process allows for a more precise understanding of the user's emotional state.

[0967] Creating a response message

[0968] The server generates empathy and counseling messages based on the analysis results and the recognized emotional state. The program uses conditional branching to select or generate appropriate messages. For example, if the condition is determined to be "stress," it generates a message saying, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help." If the emotion engine also determines that "anxiety" or "fatigue" are present, it generates additional messages accordingly.

[0969] Sending and displaying the response

[0970] The server formats the generated message into JSON format and sends it back to the device. The device receives this response and displays it to the user. The display is in text format on the device screen, and by reading the message, the user feels that their situation is understood, which may reduce their mental burden.

[0971] Examples and prompts

[0972] Below is a specific example and the prompt to input to the generative AI model.

[0973] example

[0974] User input: "I'm too busy at work and I don't have time to rest."

[0975] Generated feedback:

[0976] That must be really tough. I hope you can find some time to rest, even if just a little. If you need it, don't push yourself too hard and ask for help from those around you. I'm also concerned about your anxiety. Understand that and try to think of ways to deal with it little by little.

[0977] Prompt Sentence Examples

[0978] User input: "I'm too busy at work and I don't have time to rest."

[0979] Generative AI model input prompt:

[0980] User input: "I'm too busy at work and I don't have time to rest."

[0981] System:

[0982] The natural language processing engine analyzes the data and extracts "stress," "work," "busy," and "rest time." The emotion engine recognizes specific emotional states (e.g., stress, anxiety, fatigue) and generates appropriate messages.

[0983] This system allows users to receive psychological support relatively easily. It is also convenient because users can input their complaints and worries at any time, regardless of location or time, and receive instant feedback. The addition of an emotion engine can further enhance this effective method of maintaining mental health by recognizing the user's emotional state in more detail and providing corresponding messages.

[0984] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0985] Step 1:

[0986] A user launches an app using a mobile information device. The app displays a prompt saying "Hello. What's up?" and provides a text input field. The user enters their complaints or worries into the text input field and presses the "Send" button. Specifically, the device acquires the user's text input and prepares it for transmission. The input includes the text of the user's worries or complaints. The output is the acquired text data.

[0987] Step 2:

[0988] The device sends the user's input to the server as an HTTP POST request. This request contains the text data entered by the user. Specifically, the device sends data to the destination URL and waits for a response from the server. The input is the text data entered by the user, and the output is data sent to the server.

[0989] Step 3:

[0990] The server analyzes the text data received from the terminal. First, a natural language processing engine (e.g., spaCy) is used to perform morphological analysis and extract keywords. The text data is input for this analysis. Specifically, through morphological analysis, the keywords "work," "busy," and "time to rest" are extracted. The extracted keywords are generated as output.

[0991] Step 4:

[0992] The server uses an emotion engine (e.g., Hugging Face Transformers) to analyze the emotional state of the text data. The input is the keywords extracted in the previous step. Specifically, the emotion engine analyzes the context of the text and recognizes emotions such as "stress" and "anxiety." The output is the recognized emotional state.

[0993] Step 5:

[0994] The server generates empathy and counseling messages based on the analysis results and the recognized emotional state. The inputs are the recognized emotional state and extracted keywords. Specific operations include using conditional branching to select an appropriate message. For example, if it is determined to be "stressed," it generates a message saying, "That must be really tough. It would be good if you could find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you." The generated message is output.

[0995] Step 6:

[0996] The server formats the generated message into JSON format and sends it back to the terminal. The input is the generated message. Specifically, the generated message is converted into JSON format and an HTTP POST request is sent to the terminal again. The formatted message is generated as the output.

[0997] Step 7:

[0998] The device displays the received message to the user. The input is the received JSON format message. Specifically, the device parses the message and displays it on the screen. By reading this, the user feels that their feelings have been understood. The displayed message is generated as the output.

[0999] Through these steps, users can receive psychological support relatively easily. This system accurately grasps the user's emotional state and provides appropriate feedback to maintain mental health. It is also available at any time, regardless of location or time.

[1000] (Application example 2)

[1001] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1002] In recent years, technologies for improving consumer purchasing experiences have been attracting attention. In particular, brick-and-mortar stores are required to quickly resolve customer problems and provide appropriate support. However, current systems have difficulty accurately understanding customers' emotions and concerns and providing appropriate empathy and counseling messages. Furthermore, when using mobile information terminals to achieve this, real-time feedback and support are insufficient. This invention aims to solve these problems and quickly resolve problems customers face in brick-and-mortar stores.

[1003] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving complaints and worries input by a user using a mobile information terminal, natural language processing means for analyzing the received input data and extracting emotions and keywords, means for generating empathy and counseling messages based on the analyzed data, and means for displaying the generated messages on the mobile information terminal or the wearable display device. This makes it possible to provide immediate and appropriate support for problems users encounter in physical stores and improve customer experience.

[1004] "Mobile information terminal" refers to an electronic device that can be carried by a user and that can input and display data. Specifically, it includes smartphones and tablet terminals.

[1005] "Wearable display devices" refer to display devices that can be worn by the user. Examples include smart glasses and head-mounted displays.

[1006] "Natural language processing means" refers to technology that analyzes text data entered by users and extracts emotions and keywords. Specifically, this is done using a natural language processing engine.

[1007] "Emotion analysis engine" refers to technology that analyzes emotions contained in received text data and determines the user's emotional state, enabling a more accurate understanding of the user's emotions.

[1008] An "empathetic message" is a message that shows empathy for the user's feelings and situation and understands how the user feels, thereby reducing the user's mental burden.

[1009] A "counseling message" is a message that provides specific advice or suggestions for the problems or worries that the user is having, thereby helping the user to solve the problem.

[1010] A "server" is a computer system that receives and processes data sent by users. It uses various engines to analyze data and generate messages.

[1011] This invention is a system that receives complaints and worries input by a user using a mobile information terminal or a wearable display device, analyzes the input, and generates appropriate empathy and counseling messages. Detailed embodiments of this system are described below.

[1012] User Input

[1013] A user launches an application using a mobile information terminal (such as a smartphone or tablet) or a wearable display device (such as smart glasses). The application displays a prompt, saying, "Please let us know if you have any problems or complaints about the store," and provides a voice or text input field. For example, the user might type, "The store is too crowded and I feel uneasy. I can't even find the restroom." Then, by pressing the "Submit" button, the input data is captured by the device.

[1014] Sending data

[1015] After the terminal receives the user's input, it sends the text data to the server using an HTTP POST request and waits for a response from the server. At this time, it uses the requests module to send the data to the server.

[1016] Data analysis

[1017] The server analyzes the text data received from the device. A natural language processing engine and a sentiment analysis engine are used for the analysis, and emotions and keywords from the input text are extracted. For example, in response to an input such as "The store is too crowded and I feel uneasy. I don't know where the restroom is," the natural language processing engine extracts "stress due to crowding" as the emotion and "inside the store," "crowded," and "restroom" as keywords.

[1018] Emotion recognition

[1019] The server uses an emotion analysis engine to recognize the user's specific emotional state based on the extracted keywords and context. For example, it determines the degree of "stress due to crowding." It also determines other related emotions (e.g., "anxiety" or "dissatisfaction"), allowing for a more precise understanding of the user's emotional state.

[1020] Creating a response message

[1021] The server generates empathy and counseling messages based on the analysis results and the recognized emotional state. For example, if it determines that the customer is feeling "stressed due to crowding," it generates a message saying, "It seems you're worried about the store being crowded. The restroom is located at the back right of the store. If you wait a moment, a staff member will assist you." If the emotion analysis engine also determines that "anxiety" is present, it generates an additional message accordingly.

[1022] Sending and displaying the response

[1023] The server formats the generated message into JSON format and sends it back to the device. The device receives this response and displays it to the user. Seeing the displayed message makes the user feel that their situation is understood, which may reduce their mental burden. For example, a user who types, "The store is too crowded and I feel uneasy. I can't even find the restroom," could receive the following feedback:

[1024] Prompt Sentence Examples

[1025] User input: "The store is too crowded and I feel uneasy. I don't know where the restroom is."

[1026] Feedback message: "I understand you're concerned about the store being busy. The restroom is located at the back right of the store. Please wait a moment and a member of staff will help you."

[1027] The system allows users to receive quick support for any issues they may encounter in-store, improving their shopping experience.

[1028] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1029] Step 1:

[1030] A user launches an application using a mobile information terminal or a wearable display device. The terminal displays a prompt on the screen saying, "Please let us know if you have any problems or complaints about the store," and provides voice and text input fields. The input here is the user's complaints or worries. For example, the user might enter, "The store is too crowded and I feel uneasy. I can't even find the restroom."

[1031] input:

[1032] Text of the user's complaints or worries (e.g., "The store is too crowded and I can't relax. I don't know where the restroom is.")

[1033] output:

[1034] User input text data

[1035] Step 2:

[1036] The terminal obtains the user's input text data and sends it to the server using an HTTP POST request, using the requests module to send the data to the server.

[1037] input:

[1038] User input text data

[1039] output:

[1040] Text data sent to the server

[1041] Step 3:

[1042] The server analyzes the received text data. Using a natural language processing engine, the server extracts emotions and keywords from the input text. In this example, for the text "The store is too crowded and I feel uneasy. I don't even know where the restroom is," the server extracts "stress due to crowding" as the emotion, and extracts "inside the store," "crowded," and "restroom" as keywords.

[1043] input:

[1044] Text data sent to the server

[1045] output:

[1046] Extracted emotions and keywords

[1047] Step 4:

[1048] The server uses an emotion analysis engine to determine the user's specific emotional state based on the extracted keywords and context. For example, it evaluates the degree of "stress due to crowding" and determines related emotions. At this time, it also analyzes whether other emotions (such as "anxiety" or "dissatisfaction") are present.

[1049] input:

[1050] Extracted emotions and keywords

[1051] output:

[1052] The user's specific emotional state

[1053] Step 5:

[1054] The server generates empathy and counseling messages based on the analysis results and the recognized emotional state. For example, if it determines that the customer is feeling "stressed due to crowding," it generates a message saying, "It seems you're worried about the store being crowded. The restroom is located at the back right of the store. If you wait a moment, a staff member will assist you." If the emotion analysis engine determines that "anxiety" is also present, it generates an additional message accordingly.

[1055] input:

[1056] The user's specific emotional state

[1057] output:

[1058] Generated empathy and counseling messages

[1059] Step 6:

[1060] The server formats the generated message into JSON format and sends it back to the device. The device receives this response and displays it to the user. Seeing the displayed message can help the user feel that their situation is understood, which can reduce their mental burden.

[1061] input:

[1062] Generated empathy and counseling messages

[1063] output:

[1064] Feedback message displayed on the device

[1065] This process allows users to receive prompt and appropriate support for any issues they may have, improving their shopping experience in physical stores.

[1066] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1067] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1068] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1069] [Fourth embodiment]

[1070] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1071] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1072] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1073] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1074] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1075] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1076] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1077] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1078] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1079] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1080] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1081] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1082] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1083] The present invention is a system in which a user inputs complaints or worries using a mobile information terminal, and a server analyzes the input and generates an appropriate message. Specific embodiments for carrying out the present invention will be described below.

[1084] User Input

[1085] A user starts an application using a mobile information terminal and inputs their complaints or worries. For example, they can type something like "I'm so busy at work I don't have time to rest" directly into the text input field. Then, by pressing the "Send" button, the input data is retrieved by the terminal.

[1086] Sending data

[1087] After receiving the user's input, the terminal sends the text data to the server via an HTTP POST request. The terminal sends this request and waits for a response from the server.

[1088] Data analysis

[1089] The server analyzes the text data received from the device. A natural language processing engine is used for the analysis to extract emotions and keywords from the input text. For example, for an input such as "I'm so busy at work I don't have time to rest," the analysis engine extracts "stress" as the emotion and "work," "busy," and "time to rest" as keywords.

[1090] Creating a response message

[1091] The server generates empathy and counseling messages based on the analysis results. For example, if the analysis results indicate "stress," it generates a message such as, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you." This message provides specific empathy and practical advice for the user's situation.

[1092] Sending and displaying the response

[1093] The server formats the generated message into JSON format and sends it back to the device. The device receives this response and displays it to the user. By seeing the displayed message, the user feels that their situation is understood, which may reduce their mental burden.

[1094] For example, a user who types "I'm too busy at work and don't have time to rest" may receive feedback like this:

[1095] AI: "That's really tough. I hope you can find some time to rest, even if it's just for a little while. Don't push yourself too hard, and ask for help if you need it."

[1096] This system allows users to receive psychological support relatively easily. It is also convenient because users can input their complaints and worries at any time, regardless of location or time, and receive instant feedback. This makes it possible to provide an effective means of reducing individual stress and worries and maintaining mental health.

[1097] The processing flow will be explained below.

[1098] Step 1:

[1099] A user launches an app using a mobile device. The app displays a prompt saying "Hello, how can I help you?" and provides a text input field.

[1100] Step 2:

[1101] The user enters their complaint or worry into the text input field, for example, "I'm so busy at work I don't have time to rest." Then, the user clicks the "Send" button.

[1102] Step 3:

[1103] The device obtains user input and sends the obtained text data to the server as an HTTP POST request to a pre-configured API endpoint.

[1104] Step 4:

[1105] The server receives the HTTP request, retrieves the text data, and passes it to a natural language processing engine for analysis.

[1106] Step 5:

[1107] The server uses a natural language processing engine to analyze the text data and extract emotions and keywords. For example, for an input such as "I'm so busy at work I don't have time to rest," the server extracts "stress" as the emotion and "work," "busy," and "time to rest" as keywords.

[1108] Step 6:

[1109] The server generates empathy and counseling messages based on the analysis results. For example, if the server determines that the patient is suffering from stress, it generates a message such as, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you."

[1110] Step 7:

[1111] The server formats the generated message into JSON format and sends the formatted data to the terminal as an HTTP response.

[1112] Step 8:

[1113] The device receives the response from the server, parses the received JSON data, and extracts the message.

[1114] Step 9:

[1115] The device displays the extracted message. The user confirms the displayed message. For example, a message like "AI: That's really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you" may be displayed.

[1116] The above steps realize a system that allows users to instantly receive sympathy and counseling for their complaints and worries.

[1117] Example 1

[1118] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1119] In modern society, many people suffer from stress and worries in their work and personal lives. The lack of an environment where people can easily seek advice about these worries leads to an accumulation of mental stress, creating a problem. Furthermore, time and financial constraints mean that many people are unable to receive appropriate support from professional counseling. For this reason, there is a need for a system that allows users to easily input their complaints and worries and receive feedback on them.

[1120] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1121] In this invention, the server includes means for receiving text data from a mobile information terminal, means for transmitting the received text data as an HTTP POST request, means including a natural language processing engine for analyzing the received text data, means for generating empathy and counseling messages based on the analysis results, and means for formatting the generated messages into JSON format and transmitting them again to the mobile information terminal. This allows users to easily input their complaints and worries and receive instant feedback, thereby reducing mental stress and providing psychological support.

[1122] "User" refers to an individual who uses the system to input complaints and worries and receive feedback.

[1123] "Mobile information terminal" refers to a portable computer terminal that can connect to the Internet, such as a smartphone or tablet.

[1124] "Text data" refers to text information input by a user using a mobile information terminal.

[1125] An "HTTP POST request" is one of the HTTP protocol methods for a client to send data to a server.

[1126] "Server" refers to a computer system that receives requests from clients over a network and processes and responds to the requests.

[1127] A "natural language processing engine" refers to a software system that analyzes input text data and extracts emotions and keywords.

[1128] "Emotion" refers to the user's psychological state analyzed from text data.

[1129] "Keywords" refer to important words or phrases in the text data.

[1130] A "sympathetic message" refers to a sentence that shows understanding and sympathy for the user's input.

[1131] A "counseling message" refers to a piece of text that provides advice and support for a user's concerns.

[1132] "JSON format" is an abbreviation for JavaScript Object Notation and refers to a lightweight data format for data exchange.

[1133] The present invention is a system in which a user inputs complaints or worries using a mobile information terminal, and a server analyzes the input and generates an appropriate message. Specifically, the present invention is implemented using the following hardware and software.

[1134] Provided environment

[1135] Hardware and Software

[1136] 1. Personal digital assistant: A mobile device such as a smartphone or tablet, which includes an input / output interface (e.g., a touchscreen) and internet connectivity.

[1137] 2. Server: A powerful computer system that communicates with clients over the Internet and analyzes the data it receives.

[1138] 3. Natural Language Processing Engine: A software system used to analyze incoming text data, such as the Google Cloud Natural Language API, to extract sentiment and keywords from the text entered by the user.

[1139] 4. API Requests and Responses: HTTP POST requests and responses are used as the communication protocol for exchanging data between the mobile information terminal and the server.

[1140] Implementation method

[1141] 1. User Input

[1142] The user uses a mobile information terminal to launch a dedicated application and enter their complaints or worries into the text field. For example, they can type, "I'm so busy at work I don't have time to rest," and press the "Send" button. This operation causes the terminal to acquire the user's input data.

[1143] 2. Data transmission

[1144] The device receives the text data entered by the user and sends it to the server as an HTTP POST request, along with metadata such as the user ID and a timestamp, so the server is ready to process the data.

[1145] 3. Receiving and analyzing text data

[1146] The server receives a POST request sent from the device. The received data is decoded and passed to a natural language processing engine for sentiment analysis and keyword extraction. For example, from the text "I'm so busy at work I don't have time to rest," the emotion "stress" and the keywords "work," "busy," and "time to rest" are extracted.

[1147] 4. Creating a Response Message

[1148] The server generates empathy and counseling messages based on the analysis results. For example, if it determines that the user is experiencing stress, it generates a message such as, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you."

[1149] 5. Sending and Displaying the Response

[1150] The server formats the generated message into JSON format and sends it back to the device as an HTTP response. The device receives the response from the server, parses the received data, and displays it in the user interface. This allows the user to receive feedback and feel that their situation is understood.

[1151] Examples and prompts

[1152] If the user types "I'm too busy at work and don't have time to rest," the following feedback will be displayed:

[1153] Specific examples

[1154] User input:

[1155] "I'm so busy with work that I don't have time to rest"

[1156] Server Feedback:

[1157] "That must be really hard. I hope you can find some time to rest, even if just a little. If you need it, don't push yourself too hard and ask for help from those around you."

[1158] Prompt Sentence Examples

[1159] User input: "I'm too busy at work and I don't have time to rest."

[1160] Message to generate: A message that contains empathy and specific advice to reduce the user's stress.

[1161] In this way, we provide a system that allows users to easily input their complaints and worries on a mobile information terminal and receive appropriate feedback immediately, thereby reducing the user's mental burden and providing psychological support.

[1162] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1163] Step 1: User Input

[1164] A user starts the application using a mobile information terminal, enters their complaints or worries into the text input field, for example, "I'm so busy at work I don't have time to rest," and presses the "Send" button.

[1165] Input: User text input

[1166] Output: Text data acquired within the device when the send button is pressed

[1167] Specific behavior: The user taps the app to launch it, enters text in the text field, and presses the send button.

[1168] Step 2: Send data

[1169] The device receives the text data and sends it to the server as an HTTP POST request, which also includes metadata such as the user ID and a timestamp.

[1170] Input: Text data and metadata acquired within the device

[1171] Output: Data sent to the server as an HTTP POST request

[1172] Specific operation: The terminal stores the input data in a buffer, generates an HTTP POST request, and sends it to the server.

[1173] Step 3: Receiving text data

[1174] The server receives the HTTP POST request sent from the terminal and decodes it to extract text data and metadata.

[1175] Input: HTTP POST request from terminal

[1176] Output: Extracted text data and metadata

[1177] What it does: The server listens for HTTP requests and parses the data to get the required information.

[1178] Step 4: Analyzing the text data

[1179] The server uses a natural language processing engine to analyze the received text data and extract emotions and keywords. For example, in the case of "I'm so busy at work I don't have time to rest," the server extracts the emotion "stress" and the keywords "work," "busy," and "time to rest."

[1180] Input: Extracted text data

[1181] Output: Parsed sentiment and keywords

[1182] What happens: The server sends an API request to the natural language processing engine and runs a script to parse the results.

[1183] Step 5: Generate a response message

[1184] The server generates empathy and counseling messages based on the analysis results. For example, if the server determines that the patient is suffering from stress, it generates a message such as, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you."

[1185] Input: Parsed sentiment and keywords

[1186] Output: Generated empathy and counseling messages

[1187] Specific operation: The server executes a script that generates a message using a predefined message template based on the analysis results.

[1188] Step 6: Sending the Response

[1189] The server formats the generated message into JSON format and resends it to the terminal as an HTTP response.

[1190] Input: Generated empathy and counseling messages

[1191] Output: HTTP response in structured JSON data format

[1192] Specific operation: The server generates a message, encodes it into JSON format, and sends the data to an API endpoint that sends an HTTP response.

[1193] Step 7: Receiving and Viewing Messages

[1194] The device receives the HTTP response from the server, parses the received JSON data, and displays the parsed message in the user interface.

[1195] Input: JSON formatted HTTP response sent from the server

[1196] Output: The message that appears in the user interface

[1197] Specific behavior: The device listens for HTTP responses, parses the received data in JSON format, and displays the results in the application's UI.

[1198] These steps allow users to easily input their complaints and worries and receive instant feedback, thereby reducing mental stress and providing psychological support.

[1199] (Application example 1)

[1200] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1201] In recent years, there has been a demand for improved customer service and customer satisfaction in brick-and-mortar stores. However, due to a lack of appropriate systems for responding to minor concerns and complaints from customers, customer dissatisfaction can easily accumulate. It is also difficult to communicate effectively with customers and propose appropriate products and services.

[1202] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1203] In this invention, the server includes means for receiving complaints and worries input by a user using a mobile information terminal, natural language processing means for analyzing the received input data and extracting emotions and keywords, means for generating empathy and counseling messages based on the analyzed data, means for displaying the generated messages, means for equipping a robot with a touch screen and communicating with customers in the store, and means for proposing products and services of the store based on the customer's input. This makes it possible to provide immediate feedback on customers' worries and complaints and to propose products and services that meet the customer's needs.

[1204] A "personal digital assistant" is an electronic device that a user can carry and that allows the user to input and transmit data such as text and voice.

[1205] "Input data" refers to complaints, worries, and other text information entered by a user using a mobile information terminal or the touch screen of a robot.

[1206] "Natural language processing" refers to the technology and process for analyzing input data and extracting emotions and keywords from it.

[1207] "Empathy and counseling message" refers to a message containing empathy and advice for the user's situation, which is generated based on the analysis of data entered by the user.

[1208] "Display means" refers to techniques and devices for visually presenting the generated message to the user.

[1209] "Equipping a robot with a touchscreen" refers to attaching a touch-operable display to the exterior of the robot.

[1210] "Means of communicating with customers in-store" refers to the robot physically roaming the store and interacting with customers through touchscreens and voice.

[1211] "Means for suggesting products and services" refers to the technology and process that provides information about appropriate products and services based on customer input.

[1212] The present invention is a system for improving customer service in brick-and-mortar stores by analyzing complaints and worries entered by users using terminals, generating empathy and counseling messages, and even suggesting appropriate products and services. Specific embodiments of the present invention will be described below.

[1213] User Input

[1214] Users can use the touchscreen of the in-store robot to input their complaints and worries, for example, "I'm so busy at work I don't have time to rest." The input data is then captured by the robot.

[1215] Sending data

[1216] The text data entered by the user is sent from the robot to the server using an HTTP POST request.

[1217] Data analysis

[1218] The server analyzes the text data received from the device using a Natural Language Processing (NLP) engine (e.g., TextBlob). Through analysis, emotions and keywords from the input text are extracted. For example, for an input such as "I'm so busy at work I don't have time to rest," the emotion extracted is "stress," and the keywords extracted are "work," "busy," and "time to rest."

[1219] Creating a response message

[1220] The server generates empathy and counseling messages based on the analysis results. For example, if the analysis results indicate "stress," it generates a message such as, "That's really tough. It's important to take a break." This message shows specific empathy for the user's situation and provides practical advice.

[1221] Sending and displaying the response

[1222] The generated message is formatted in JSON format and sent back to the robot. The robot receives this message and displays it to the user. By seeing the displayed message, the user can feel that their situation is understood, which may reduce their mental burden.

[1223] Product and service suggestions

[1224] Furthermore, the system can also suggest products and services from stores related to the complaints and worries received. For example, if a user inputs that they are "too busy at work and don't have time to rest," the system can suggest products and services that have a relaxing effect.

[1225] Specific examples

[1226] Prompt Sentence Example 1:

[1227] Enter: "I'm so busy at work I don't have time to rest."

[1228] Response: "That's really tough. It's important to get some rest."

[1229] Prompt Sentence Example 2:

[1230] Enter: "I've been feeling down lately."

[1231] Response: "I get that sometimes. I'm happy to help."

[1232] The server analyzes the data entered by the user, generates optimal messages, and suggests related products and services. This system can increase customer satisfaction and improve the quality of customer service in stores.

[1233] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1234] Step 1:

[1235] The user inputs their complaints and worries into the robot's touchscreen. The user inputs the text "I'm too busy at work and I don't have time to rest" into the robot's touchscreen. The input data is captured by the terminal.

[1236] Step 2:

[1237] The device sends the user's input data to the server using an HTTP POST request. The device converts the input text data into JSON format and sends it to the server.

[1238] Step 3:

[1239] The server receives the HTTP POST request and analyzes the input data. The server uses a Natural Language Processing (NLP) engine (e.g., TextBlob) to extract emotions and keywords from the input text. The input is the text data sent by the user, and the output is the emotion "stress" and keywords such as "work," "busy," and "time to rest."

[1240] Step 4:

[1241] The server generates empathy and counseling messages based on the analysis results. Using a generative AI model, it creates a message for the emotion "stress" such as "That's really tough. It's important to take a break." The input is the analysis results, and the output is the generated empathy and counseling message.

[1242] Step 5:

[1243] The server formats the generated message in JSON format and sends it back to the device. The input is the empathy and counseling message, and the output is the message formatted in JSON format.

[1244] Step 6:

[1245] The device displays the message received from the server. Specifically, the user can see the message "This is really hard work. It's important to take a break." on the robot's touchscreen.

[1246] Step 7:

[1247] The server proposes products and services from the store that are related to the customer's complaints and worries. For example, it provides information on products (such as relaxation goods) and services (such as relaxation services) related to "stress" and "fatigue." The input is emotions and related keywords, and the output is related product proposal information.

[1248] Step 8:

[1249] The terminal displays suggested products and services to the user, who can check suggestions such as "relaxation goods" and "relaxation services" on the robot's touchscreen.

[1250] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1251] The present invention is a system that receives complaints and worries input by a user using a mobile information terminal, and a server analyzes the input and generates an appropriate message. Furthermore, by combining it with an emotion engine, the system can more accurately determine the user's emotional state and provide individual empathy and counseling messages. Specific embodiments for implementing the present invention are described below.

[1252] User Input

[1253] A user launches an app using a mobile information device. The app displays a prompt, "Hello. What's up?" and provides a text input field. The user enters their complaints or worries into the text input field. For example, they might enter, "I'm so busy at work, I don't have time to rest." Then, by pressing the "Send" button, the entered data is retrieved by the device.

[1254] Sending data

[1255] After receiving the user's input, the terminal sends the text data to the server using an HTTP POST request and waits for a response from the server.

[1256] Data analysis

[1257] The server analyzes the text data received from the device. For analysis, it uses a natural language processing engine and an emotion engine to extract emotions and keywords from the input text. For example, for an input such as "I'm so busy at work I don't have time to rest," the natural language processing engine extracts "stress" as the emotion and "work," "busy," and "time to rest" as keywords.

[1258] Emotion recognition

[1259] The server uses an emotion engine to recognize the user's specific emotional state based on the extracted keywords and context, for example, determining the level of "stress" and other related emotions (e.g., "anxiety" or "fatigue"). This allows for a more precise understanding of the user's emotional state.

[1260] Creating a response message

[1261] The server generates empathy and counseling messages based on the analysis results and the recognized emotional state. For example, if it determines that the user is experiencing "stress," it generates a message like, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you." Furthermore, if the emotion engine determines that "anxiety" or "fatigue" are also present, it generates additional messages accordingly.

[1262] Sending and displaying the response

[1263] The server formats the generated message into JSON format and sends it back to the device. The device receives this response and displays it to the user. By seeing the displayed message, the user feels that their situation is understood, which may reduce their mental burden.

[1264] For example, a user who types "I'm too busy at work and don't have time to rest" may receive feedback like this:

[1265] AI: "That's really tough. I hope you can find some time to rest, even if just a little. If you need it, don't push yourself too hard and ask for help. I'm also concerned about your anxiety. I understand that and we can try to find some solutions, little by little."

[1266] This system allows users to receive psychological support relatively easily. It is also convenient because users can input their complaints and worries at any time, regardless of location or time, and receive instant feedback. The addition of an emotion engine can further enhance this effective method of maintaining mental health by recognizing the user's emotional state in more detail and providing corresponding messages.

[1267] The processing flow will be explained below.

[1268] Step 1:

[1269] A user launches an app using a mobile device. The app displays a prompt saying "Hello, how can I help you?" and provides a text input field.

[1270] Step 2:

[1271] The user enters their complaints or worries into the text input field, for example, "I'm so busy at work I don't have time to rest," and then clicks the "Send" button.

[1272] Step 3:

[1273] The device obtains user input and sends the obtained text data to the server as an HTTP POST request to a pre-configured API endpoint.

[1274] Step 4:

[1275] The server receives the HTTP request and retrieves the text data, which is then passed to a natural language processing engine to extract sentiment and keywords.

[1276] Step 5:

[1277] The server uses a natural language processing engine to analyze the text data. For example, for an input such as "I'm so busy at work I don't have time to rest," it extracts "stress" as the emotion and "work," "busy," and "time to rest" as keywords.

[1278] Step 6:

[1279] The server uses an emotion engine to recognize the user's specific emotional state based on the extracted keywords and context, for example, determining the degree of "stress" and other related emotions (e.g., "anxiety" or "fatigue").

[1280] Step 7:

[1281] The server generates empathy and counseling messages based on the analysis results and the recognition results of the emotion engine. For example, if it determines that the symptom is "stress" and "anxiety" is also present, it generates a message saying, "That must be really tough. It would be good if you could find some time to rest. If necessary, try not to push yourself and ask for help from those around you. I'm also concerned about your anxiety. Let's understand that and try to think of ways to deal with it little by little."

[1282] Step 8:

[1283] The server formats the generated message into JSON format and sends the formatted data to the terminal as an HTTP response.

[1284] Step 9:

[1285] The device receives the response from the server, parses the received JSON data, and extracts the message.

[1286] Step 10:

[1287] The device displays the extracted message. The user confirms the displayed message. For example, a message might appear that reads, "AI: That must be really tough. I hope you can find some time to rest, even if just a little. If necessary, don't push yourself too hard and ask for help from those around you. I'm also concerned about your anxiety. I understand that, and let's try to come up with solutions little by little."

[1288] The above steps will realize a system that allows users to receive instant empathy and counseling for their complaints and worries. The addition of an emotion engine will enable a deeper understanding of the user's emotional state and provide personalized feedback accordingly.

[1289] Example 2

[1290] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1291] In modern society, mental strain caused by stress, anxiety, and constant busy schedules is a common problem for many people. However, in our busy daily lives, opportunities to appropriately share these concerns and receive professional feedback are limited. Furthermore, conventional systems have had difficulty accurately recognizing a user's emotions and providing individually tailored feedback. The present invention aims to provide a system that analyzes complaints and worries entered by a user, accurately grasps their emotional state, and provides empathy and counseling messages.

[1292] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for analyzing received input data using a natural language processing engine and an emotion engine and extracting emotions and keywords, a means for generating empathy and counseling messages based on the analyzed data, and a means for formatting the generated messages in JSON format and transmitting them to the mobile information terminal. This makes it possible to grasp the user's emotional state in more detail and provide appropriate feedback according to the individual situation.

[1293] A "personal digital assistant" is a mobile device used by a user, such as a smartphone or tablet, that is capable of text input and communication.

[1294] "Complaints and worries" are matters related to dissatisfaction, worries, stress, and difficulties that users feel in their daily lives and at work.

[1295] A "server" is a computer system that receives, analyzes, and processes data sent by users.

[1296] "Means for receiving" refers to the process or technology for transferring data from a mobile information terminal to a server, and includes, for example, communication technology using an HTTP POST request.

[1297] A "natural language processing engine" is software for analyzing text data and extracting grammatical and semantic information, and has functions such as morphological analysis and keyword extraction.

[1298] An "emotion engine" is software that analyzes emotional states from text data and recognizes the classification and intensity of emotions.

[1299] "Means for analyzing" refers to the process of processing received data using a natural language processing engine and an emotion engine to extract necessary information.

[1300] The "empathy and counseling message" is a feedback message generated according to the user's emotions and circumstances, and includes content that sympathizes with the user's feelings and provides appropriate advice and comfort.

[1301] "Generative means" refers to the processes and techniques for constructing empathy and counseling messages based on the analyzed data.

[1302] The "JSON format" is a lightweight data format for exchanging data that is easy to read for both humans and machines.

[1303] "Means for sending" refers to the process for delivering the generated message to the mobile information terminal, and includes, for example, data communication via the Internet.

[1304] "Displaying means" refers to the process or technique for visually presenting a message to a user on a mobile device.

[1305] MODE FOR CARRYING OUT THE INVENTION

[1306] This system receives complaints and worries entered by users using a mobile information terminal, and the server analyzes them to generate appropriate messages. Furthermore, by combining it with an emotion engine, the system can more accurately determine the user's emotional state and provide personalized empathy and counseling messages.

[1307] Specific Embodiments

[1308] User Input

[1309] A user launches an app using a mobile information device. The app displays a prompt, "Hello. What's up?" and provides a text input field. The user enters their complaints or worries into the text input field. For example, they might enter, "I'm so busy at work I don't have time to rest." Then, by pressing the "Send" button, the entered data is retrieved by the device.

[1310] Sending data

[1311] After receiving the user's input, the device sends the text data to the server using an HTTP POST request and waits for a response from the server. The destination URL is set in advance.

[1312] Data analysis

[1313] The server analyzes the text data received from the device. For analysis, it uses a Python-based natural language processing engine (e.g., spaCy) and an emotion engine (e.g., Hugging Face Transformers). It performs morphological analysis of the input text and extracts keywords. For example, for the input "I'm so busy at work I don't have time to rest," the natural language processing engine extracts "stress" as the emotion and "work," "busy," and "time to rest" as keywords.

[1314] Emotion recognition

[1315] The server uses an emotion engine to recognize the user's specific emotional state based on the extracted keywords and context. The emotion engine uses the BERT model to determine the level of "stress" and other related emotions (e.g., "anxiety" or "fatigue"). This process allows for a more precise understanding of the user's emotional state.

[1316] Creating a response message

[1317] The server generates empathy and counseling messages based on the analysis results and the recognized emotional state. The program uses conditional branching to select or generate appropriate messages. For example, if the condition is determined to be "stress," it generates a message saying, "That must be really tough. I hope you can find some time to rest. If necessary, don't push yourself too hard and ask for help." If the emotion engine also determines that "anxiety" or "fatigue" are present, it generates additional messages accordingly.

[1318] Sending and displaying the response

[1319] The server formats the generated message into JSON format and sends it back to the device. The device receives this response and displays it to the user. The display is in text format on the device screen, and by reading the message, the user feels that their situation is understood, which may reduce their mental burden.

[1320] Examples and prompts

[1321] Below is a specific example and the prompt to input to the generative AI model.

[1322] example

[1323] User input: "I'm too busy at work and I don't have time to rest."

[1324] Generated feedback:

[1325] That must be really tough. I hope you can find some time to rest, even if just a little. If you need it, don't push yourself too hard and ask for help from those around you. I'm also concerned about your anxiety. Understand that and try to think of ways to deal with it little by little.

[1326] Prompt Sentence Examples

[1327] User input: "I'm too busy at work and I don't have time to rest."

[1328] Generative AI model input prompt:

[1329] User input: "I'm too busy at work and I don't have time to rest."

[1330] System:

[1331] The natural language processing engine analyzes the data and extracts "stress," "work," "busy," and "rest time." The emotion engine recognizes specific emotional states (e.g., stress, anxiety, fatigue) and generates appropriate messages.

[1332] This system allows users to receive psychological support relatively easily. It is also convenient because users can input their complaints and worries at any time, regardless of location or time, and receive instant feedback. The addition of an emotion engine can further enhance this effective method of maintaining mental health by recognizing the user's emotional state in more detail and providing corresponding messages.

[1333] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1334] Step 1:

[1335] A user launches an app using a mobile information device. The app displays a prompt saying "Hello. What's up?" and provides a text input field. The user enters their complaints or worries into the text input field and presses the "Send" button. Specifically, the device acquires the user's text input and prepares it for transmission. The input includes the text of the user's worries or complaints. The output is the acquired text data.

[1336] Step 2:

[1337] The device sends the user's input to the server as an HTTP POST request. This request contains the text data entered by the user. Specifically, the device sends data to the destination URL and waits for a response from the server. The input is the text data entered by the user, and the output is data sent to the server.

[1338] Step 3:

[1339] The server analyzes the text data received from the terminal. First, a natural language processing engine (e.g., spaCy) is used to perform morphological analysis and extract keywords. The text data is input for this analysis. Specifically, through morphological analysis, the keywords "work," "busy," and "time to rest" are extracted. The extracted keywords are generated as output.

[1340] Step 4:

[1341] The server uses an emotion engine (e.g., Hugging Face Transformers) to analyze the emotional state of the text data. The input is the keywords extracted in the previous step. Specifically, the emotion engine analyzes the context of the text and recognizes emotions such as "stress" and "anxiety." The output is the recognized emotional state.

[1342] Step 5:

[1343] The server generates empathy and counseling messages based on the analysis results and the recognized emotional state. The inputs are the recognized emotional state and extracted keywords. Specific operations include using conditional branching to select an appropriate message. For example, if it is determined to be "stressed," it generates a message saying, "That must be really tough. It would be good if you could find some time to rest. If necessary, don't push yourself too hard and ask for help from those around you." The generated message is output.

[1344] Step 6:

[1345] The server formats the generated message into JSON format and sends it back to the terminal. The input is the generated message. Specifically, the generated message is converted into JSON format and an HTTP POST request is sent to the terminal again. The formatted message is generated as the output.

[1346] Step 7:

[1347] The device displays the received message to the user. The input is the received JSON format message. Specifically, the device parses the message and displays it on the screen. By reading this, the user feels that their feelings have been understood. The displayed message is generated as the output.

[1348] Through these steps, users can receive psychological support relatively easily. This system accurately grasps the user's emotional state and provides appropriate feedback to maintain mental health. It is also available at any time, regardless of location or time.

[1349] (Application example 2)

[1350] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1351] In recent years, technologies for improving consumer purchasing experiences have been attracting attention. In particular, brick-and-mortar stores are required to quickly resolve customer problems and provide appropriate support. However, current systems have difficulty accurately understanding customers' emotions and concerns and providing appropriate empathy and counseling messages. Furthermore, when using mobile information terminals to achieve this, real-time feedback and support are insufficient. This invention aims to solve these problems and quickly resolve problems customers face in brick-and-mortar stores.

[1352] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving complaints and worries input by a user using a mobile information terminal, natural language processing means for analyzing the received input data and extracting emotions and keywords, means for generating empathy and counseling messages based on the analyzed data, and means for displaying the generated messages on the mobile information terminal or the wearable display device. This makes it possible to provide immediate and appropriate support for problems users encounter in physical stores and improve customer experience.

[1353] "Mobile information terminal" refers to an electronic device that can be carried by a user and that can input and display data. Specifically, it includes smartphones and tablet terminals.

[1354] "Wearable display devices" refer to display devices that can be worn by the user. Examples include smart glasses and head-mounted displays.

[1355] "Natural language processing means" refers to technology that analyzes text data entered by users and extracts emotions and keywords. Specifically, this is done using a natural language processing engine.

[1356] "Emotion analysis engine" refers to technology that analyzes emotions contained in received text data and determines the user's emotional state, enabling a more accurate understanding of the user's emotions.

[1357] An "empathetic message" is a message that shows empathy for the user's feelings and situation and understands how the user feels, thereby reducing the user's mental burden.

[1358] A "counseling message" is a message that provides specific advice or suggestions for the problems or worries that the user is having, thereby helping the user to solve the problem.

[1359] A "server" is a computer system that receives and processes data sent by users. It uses various engines to analyze data and generate messages.

[1360] This invention is a system that receives complaints and worries input by a user using a mobile information terminal or a wearable display device, analyzes the input, and generates appropriate empathy and counseling messages. Detailed embodiments of this system are described below.

[1361] User Input

[1362] A user launches an application using a mobile information terminal (such as a smartphone or tablet) or a wearable display device (such as smart glasses). The application displays a prompt, saying, "Please let us know if you have any problems or complaints about the store," and provides a voice or text input field. For example, the user might type, "The store is too crowded and I feel uneasy. I can't even find the restroom." Then, by pressing the "Submit" button, the input data is captured by the device.

[1363] Sending data

[1364] After the terminal receives the user's input, it sends the text data to the server using an HTTP POST request and waits for a response from the server. At this time, it uses the requests module to send the data to the server.

[1365] Data analysis

[1366] The server analyzes the text data received from the device. A natural language processing engine and a sentiment analysis engine are used for the analysis, and emotions and keywords from the input text are extracted. For example, in response to an input such as "The store is too crowded and I feel uneasy. I don't know where the restroom is," the natural language processing engine extracts "stress due to crowding" as the emotion and "inside the store," "crowded," and "restroom" as keywords.

[1367] Emotion recognition

[1368] The server uses an emotion analysis engine to recognize the user's specific emotional state based on the extracted keywords and context. For example, it determines the degree of "stress due to crowding." It also determines other related emotions (e.g., "anxiety" or "dissatisfaction"), allowing for a more precise understanding of the user's emotional state.

[1369] Creating a response message

[1370] The server generates empathy and counseling messages based on the analysis results and the recognized emotional state. For example, if it determines that the customer is feeling "stressed due to crowding," it generates a message saying, "It seems you're worried about the store being crowded. The restroom is located at the back right of the store. If you wait a moment, a staff member will assist you." If the emotion analysis engine also determines that "anxiety" is present, it generates an additional message accordingly.

[1371] Sending and displaying the response

[1372] The server formats the generated message into JSON format and sends it back to the device. The device receives this response and displays it to the user. Seeing the displayed message makes the user feel that their situation is understood, which may reduce their mental burden. For example, a user who types, "The store is too crowded and I feel uneasy. I can't even find the restroom," could receive the following feedback:

[1373] Prompt Sentence Examples

[1374] User input: "The store is too crowded and I feel uneasy. I don't know where the restroom is."

[1375] Feedback message: "I understand you're concerned about the store being busy. The restroom is located at the back right of the store. Please wait a moment and a member of staff will help you."

[1376] The system allows users to receive quick support for any issues they may encounter in-store, improving their shopping experience.

[1377] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1378] Step 1:

[1379] A user launches an application using a mobile information terminal or a wearable display device. The terminal displays a prompt on the screen saying, "Please let us know if you have any problems or complaints about the store," and provides voice and text input fields. The input here is the user's complaints or worries. For example, the user might enter, "The store is too crowded and I feel uneasy. I can't even find the restroom."

[1380] input:

[1381] Text of the user's complaints or worries (e.g., "The store is too crowded and I can't relax. I don't know where the restroom is.")

[1382] output:

[1383] User input text data

[1384] Step 2:

[1385] The terminal obtains the user's input text data and sends it to the server using an HTTP POST request, using the requests module to send the data to the server.

[1386] input:

[1387] User input text data

[1388] output:

[1389] Text data sent to the server

[1390] Step 3:

[1391] The server analyzes the received text data. Using a natural language processing engine, the server extracts emotions and keywords from the input text. In this example, for the text "The store is too crowded and I feel uneasy. I don't even know where the restroom is," the server extracts "stress due to crowding" as the emotion, and extracts "inside the store," "crowded," and "restroom" as keywords.

[1392] input:

[1393] Text data sent to the server

[1394] output:

[1395] Extracted emotions and keywords

[1396] Step 4:

[1397] The server uses an emotion analysis engine to determine the user's specific emotional state based on the extracted keywords and context. For example, it evaluates the degree of "stress due to crowding" and determines related emotions. At this time, it also analyzes whether other emotions (such as "anxiety" or "dissatisfaction") are present.

[1398] input:

[1399] Extracted emotions and keywords

[1400] output:

[1401] The user's specific emotional state

[1402] Step 5:

[1403] The server generates empathy and counseling messages based on the analysis results and the recognized emotional state. For example, if it determines that the customer is feeling "stressed due to crowding," it generates a message saying, "It seems you're worried about the store being crowded. The restroom is located at the back right of the store. If you wait a moment, a staff member will assist you." If the emotion analysis engine determines that "anxiety" is also present, it generates an additional message accordingly.

[1404] input:

[1405] The user's specific emotional state

[1406] output:

[1407] Generated empathy and counseling messages

[1408] Step 6:

[1409] The server formats the generated message into JSON format and sends it back to the device. The device receives this response and displays it to the user. Seeing the displayed message can help the user feel that their situation is understood, which can reduce their mental burden.

[1410] input:

[1411] Generated empathy and counseling messages

[1412] output:

[1413] Feedback message displayed on the device

[1414] This process allows users to receive prompt and appropriate support for any issues they may have, improving their shopping experience in physical stores.

[1415] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1416] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1417] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1418] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1419] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1420] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1421] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1422] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1423] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1424] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1425] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1426] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1427] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1428] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1429] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1430] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1431] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1432] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1433] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1434] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1435] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1436] The following is further disclosed regarding the above embodiment.

[1437] (Claim 1)

[1438] means for receiving complaints and worries input by a user using a mobile information terminal;

[1439] natural language processing means for analyzing received input data and extracting emotions and keywords;

[1440] means for generating empathy and counseling messages based on the analyzed data;

[1441] a means for displaying the generated message;

[1442] A system including:

[1443] (Claim 2)

[1444] means for transmitting text data from the mobile information terminal to a server;

[1445] a natural language processing engine for processing the data received by the server;

[1446] The system of claim 1 further comprising:

[1447] (Claim 3)

[1448] A means for dynamically generating empathy messages and counseling messages according to the analysis results of the received data;

[1449] A means for formatting the generated message in JSON format and sending it to a mobile information terminal;

[1450] The system of claim 1 further comprising:

[1451] "Example 1"

[1452] (Claim 1)

[1453] means for receiving text data input by a user using a mobile information terminal;

[1454] A means for sending the received input data to the server via an HTTP POST request;

[1455] natural language processing means for analyzing the text data received by the server and extracting emotions and keywords;

[1456] means for generating empathy and counseling messages based on the analyzed data;

[1457] A means to format the generated message into JSON format and send it back to the mobile information terminal;

[1458] means for displaying the sent message on a user interface of the mobile information terminal;

[1459] A system including:

[1460] (Claim 2)

[1461] means for transmitting text data from the mobile information terminal to a server;

[1462] a natural language processing engine for processing the data received by the server;

[1463] The system of claim 1 further comprising:

[1464] (Claim 3)

[1465] A means for dynamically generating empathy messages and counseling messages according to the analysis results of the received data;

[1466] means for transmitting the generated message to a mobile information terminal and formatting the message for display to a user;

[1467] The system of claim 1 further comprising:

[1468] "Application Example 1"

[1469] (Claim 1)

[1470] means for receiving complaints and worries input by a user using a mobile information terminal;

[1471] natural language processing means for analyzing received input data and extracting emotions and keywords;

[1472] means for generating empathy and counseling messages based on the analyzed data;

[1473] a means for displaying the generated message;

[1474] The robot will be equipped with a touch screen, allowing it to communicate with customers in the store.

[1475] A means to suggest store products and services based on customer input;

[1476] A system including:

[1477] (Claim 2)

[1478] means for transmitting text data from the mobile information terminal to a server;

[1479] a natural language processing engine for processing the data received by the server;

[1480] The system of claim 1 further comprising:

[1481] (Claim 3)

[1482] A means for dynamically generating empathy messages and counseling messages according to the analysis results of the received data;

[1483] A means for formatting the generated message in JSON format and sending it to a mobile information terminal;

[1484] The system of claim 1 further comprising:

[1485] "Example 2: Combining Emotion Engines"

[1486] (Claim 1)

[1487] means for receiving complaints and worries input by a user using a mobile information terminal;

[1488] means for transmitting the received input data to a server;

[1489] A means for analyzing received input data using a natural language processing engine and an emotion engine to extract emotions and keywords;

[1490] means for generating empathy and counseling messages based on the analyzed data;

[1491] A means to format the generated message in JSON format and send it to a mobile information terminal;

[1492] means for displaying the message received by the mobile information terminal;

[1493] A system including:

[1494] (Claim 2)

[1495] means for transmitting text data from the mobile information terminal to a server;

[1496] a natural language processing engine and an emotion engine for processing the data received by the server;

[1497] The system of claim 1 further comprising:

[1498] (Claim 3)

[1499] A means for dynamically generating empathy messages and counseling messages according to the analysis results of the received data;

[1500] A means for formatting the generated message in JSON format and sending it to a mobile information terminal;

[1501] The system of claim 1 further comprising:

[1502] "Application example 2 when combining emotion engines"

[1503] (Claim 1)

[1504] means for receiving complaints and worries input by a user using a mobile information terminal;

[1505] natural language processing means for analyzing received input data and extracting emotions and keywords;

[1506] means for generating empathy and counseling messages based on the analyzed data;

[1507] a means for displaying the generated message on a portable information terminal or a wearable display device;

[1508] A system including:

[1509] (Claim 2)

[1510] means for transmitting text data from the mobile information terminal to a server;

[1511] a natural language processing engine for processing the data received by the server;

[1512] means for determining the emotional state of a user using an emotion analysis engine;

[1513] The system of claim 1 further comprising:

[1514] (Claim 3)

[1515] A means for dynamically generating empathy messages and counseling messages according to the analysis results of the received data;

[1516] means for formatting the generated message in JSON format and transmitting it to a mobile information terminal or a wearable display device;

[1517] The system of claim 1 further comprising: [Explanation of symbols]

[1518] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving complaints and worries input by a user using a mobile information terminal; natural language processing means for analyzing received input data and extracting emotions and keywords; means for generating empathy and counseling messages based on the analyzed data; a means for displaying the generated message; A system including:

2. means for transmitting text data from the mobile information terminal to a server; a natural language processing engine for processing the data received by the server; The system of claim 1 further comprising:

3. A means for dynamically generating empathy messages and counseling messages according to the analysis results of the received data; A means for formatting the generated message in JSON format and sending it to a mobile information terminal; The system of claim 1 further comprising:

Images