System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving an input from a user; means for analyzing the received input and identifying a mental state of the user; means for suggesting an appropriate relaxation method based on a result of the analysis; means for searching for a service provider related to the suggested relaxation method; means for presenting information on the service provider to the user; means for receiving and analyzing feedback from the user; and means for improving accuracy of suggesting a relaxation method based on the feedback.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, many people suffer from stress and mental disorders. However, many people feel reluctant to visit a mental clinic or psychosomatic medicine department. There is a need for a way to easily find effective stress relief methods while protecting privacy. To achieve this, a system is needed that allows users to easily consult and find appropriate relaxation methods and stress relief activities. [Means for solving the problem]

[0005] To solve the above-mentioned problems, the present invention provides a system including: means for receiving input from a user; means for analyzing the received input and identifying the user's mental state; means for proposing appropriate relaxation methods based on the analysis results; means for searching for service providers related to the proposed relaxation methods; means for presenting information about service providers to the user; means for receiving and analyzing feedback from the user; and means for improving the accuracy of relaxation method suggestions based on the feedback. This allows users to easily find the optimal stress relief method for themselves without being noticed by those around them. Furthermore, by identifying and presenting nearby service providers based on the user's location information, the system allows users to take immediate action. Furthermore, by automatically analyzing feedback and reflecting the results in the system's learning, the accuracy of suggestions will also improve in the future.

[0006] "Means for receiving input from a user" refers to an interface that allows the system to capture information entered by the user via text, voice, or other means.

[0007] "Means for analyzing received input content and identifying the user's mental state" refers to means for analyzing received input content using techniques such as natural language processing, and using the results to identify the user's current mental state and problems.

[0008] "Means for suggesting appropriate relaxation methods based on the analysis results" refers to means for presenting appropriate relaxation methods based on an algorithm in accordance with the analyzed mental state of the user.

[0009] "Means for searching for service providers related to the proposed relaxation method" refers to means for identifying affiliated service providers (e.g., aromatherapy salons) through database or online search in order to implement the proposed relaxation method.

[0010] The term "means for presenting information about service providers to users" refers to means for displaying information about service providers identified by a search in an easy-to-understand manner for users.

[0011] The "means for receiving and analyzing feedback from the user" is a means for receiving as input the effects and impressions of the relaxation method that the user has implemented, and analyzing the content thereof.

[0012] "Means for improving the accuracy of relaxation method suggestions based on feedback" refers to a means for reflecting the analysis results of received feedback in the system's learning algorithm, thereby improving the accuracy of suggestions from the next time onwards. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] This invention provides a system that proposes appropriate relaxation methods and stress-relief activities for users with mental health concerns. This system analyzes user input, proposes relaxation methods based on the results, and searches for and presents related service providers.

[0035] First, the user inputs into the device the stress, anxiety, and specific worries they experience in their daily lives. The device then sends this input as text data to the server. The server then analyzes the received text data using natural language processing technology. Specifically, it tokenizes the input and extracts keywords.

[0036] For example, consider the case where a user enters, "Recently, I've been busy at work and feeling stressed." In this case, the server extracts keywords such as "work" and "stress" and identifies the user's mental state as "caused by stress from work."

[0037] Next, the server executes a means to suggest relaxation methods based on the identified mental state. For example, it lists several relaxation methods as candidates, such as "deep breathing," "aromatherapy," and "yoga." This list prioritizes those most relevant to the user's input.

[0038] The server then executes a search for service providers related to the proposed relaxation method, identifying nearby service providers based on the user's location information from affiliated databases and online resources, thereby providing the user with information on nearby aromatherapy salons, yoga classes, relaxation facilities, etc.

[0039] The terminal visually presents the information about the service provider to the user. For example, the terminal displays a text message such as "Click here for the nearest aromatherapy salon" and a link on the user's screen.

[0040] After a user visits a salon or class and tries out a relaxation method, the device receives feedback about the effects and impressions of the experience. For example, if a user types, "I tried aromatherapy and it helped me relax a little," the device sends this feedback to the server.

[0041] Finally, the server analyzes the received feedback and applies it to the system's learning algorithm, improving the accuracy of future relaxation suggestions and making them more tailored to the user's needs.

[0042] In this way, the present invention is a system that receives input from the user, analyzes it, and suggests appropriate relaxation methods and affiliated businesses, providing a means to easily relieve stress without being noticed by those around you.

[0043] The processing flow will be explained below.

[0044] Step 1:

[0045] The user inputs information about their stress or anxiety. For example, they might input, "Work has been busy lately and I'm feeling stressed."

[0046] Step 2:

[0047] The terminal receives input from the user as text data.

[0048] Step 3:

[0049] The terminal transmits the received text data to the server.

[0050] Step 4:

[0051] The server receives the text data sent from the terminal.

[0052] Step 5:

[0053] The server uses natural language processing technology to analyze the text data and extract key keywords. In this case, the keywords extracted are "work" and "stress."

[0054] Step 6:

[0055] The server identifies the user's mental state based on the extracted keywords. In this case, "work-related stress" is identified.

[0056] Step 7:

[0057] The server then lists appropriate relaxation methods based on the identified mental state, such as "deep breathing," "aromatherapy," and "yoga."

[0058] Step 8:

[0059] The server searches for service providers related to the selected relaxation method, searching a database for nearby aromatherapy salons and yoga classes based on the user's location information.

[0060] Step 9:

[0061] The server prepares information about the service providers obtained as a result of the search and transmits it to the terminal.

[0062] Step 10:

[0063] The terminal receives the information about the service provider sent from the server.

[0064] Step 11:

[0065] The terminal displays the received information about the service provider to the user, for example, providing a text such as "Click here for nearby aromatherapy salons" and a link.

[0066] Step 12:

[0067] The user then takes action based on the displayed information about relaxation methods and service providers, for example, visiting a nearby aromatherapy salon.

[0068] Step 13:

[0069] After performing the relaxation method, the user inputs the results into the terminal as feedback, for example, "I tried aromatherapy and it helped me relax a little."

[0070] Step 14:

[0071] The terminal receives feedback from the user and transmits it to the server.

[0072] Step 15:

[0073] The server receives the feedback sent from the terminal.

[0074] Step 16:

[0075] The server analyzes the feedback content and evaluates the effectiveness of the relaxation method.

[0076] Step 17:

[0077] The server reflects the analysis results in the system's learning algorithm, improving the accuracy of proposals from next time onwards.

[0078] Through this series of steps, users can easily find the relaxation method that best suits them and effectively relieve stress.

[0079] Example 1

[0080] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0081] In recent years, the number of people suffering from stress and anxiety has been increasing, but there is a lack of specific relaxation methods to appropriately alleviate these stresses and a means to quickly obtain information on support service providers. Furthermore, there is also a lack of mechanisms to provide feedback on the effectiveness of provided relaxation methods and improve the accuracy of suggestions for future sessions. Therefore, there is a need for a system that not only analyzes user input, suggests appropriate relaxation methods, and finds corresponding service providers, but also analyzes feedback to improve services.

[0082] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0083] In this invention, the server includes means for tokenizing the received input content, extracting keywords, and identifying the user's mental state, means for listing and proposing appropriate relaxation methods based on the extracted keywords and analysis results, and means for searching for service providers related to the proposed relaxation methods using databases and online resources. This allows users to instantly obtain relaxation methods and information on service providers that are suitable for them, and the accuracy of the suggestions is further improved through feedback analysis.

[0084] "User" refers to an individual who uses the system to provide input about stress and anxiety and receive information about suggested relaxation techniques and services.

[0085] "Input content" is text data provided by the user to the system, and mainly includes information related to stress and anxiety.

[0086] "Tokenization" is the process of using natural language processing techniques to break down input content into smaller units (tokens) and convert them into a form that is easier to analyze.

[0087] "Keywords" are important words or phrases extracted from text data that serve as clues to identifying the user's mental state and required relaxation methods.

[0088] "Relaxation techniques" refer to specific measures or activities that are used to reduce a user's stress or anxiety. Examples include deep breathing, aromatherapy, and yoga.

[0089] "Service provider" refers to the facility or professional who implements the proposed relaxation method, such as a salon or classroom.

[0090] A "database" is a collection of information in which the system stores information on relaxation methods and service providers.

[0091] "Online Resources" means data and information accessible via the Internet that may be used to locate a Service Provider.

[0092] "Visually presenting" refers to displaying information in a way that users can understand visually, and primarily includes text, links, maps, etc.

[0093] "Feedback" refers to the effects and impressions of the relaxation method the user has tried, and serves as data to improve the accuracy of suggestions from next time onwards.

[0094] A "machine learning model" is an algorithm that allows the system to learn from data and improve the accuracy of relaxation method suggestions from the next time onwards.

[0095] This invention is a system that suggests relaxation methods and stress-relieving activities to alleviate the mental health concerns of users. This system has the function of analyzing the user's input, suggesting relaxation methods based on the results, and searching for and presenting related service providers.

[0096] Hardware Configuration

[0097] The main hardware required to implement the system is as follows:

[0098] User device: A device through which a user inputs information about stress and anxiety, such as a smartphone or computer.

[0099] Server: A central processing unit for analyzing input information and suggesting relaxation methods and service providers.

[0100] Software Configuration

[0101] The main software required to implement the system is as follows:

[0102] Natural language processing libraries: Used to analyze input text data. Specific examples include spaCy and NLTK.

[0103] Machine learning model: Used to analyze feedback and improve the accuracy of relaxation method suggestions. Specifically, we use scikit-learn and TensorFlow.

[0104] Database: Stores information about service providers related to the proposed relaxation methods. Specifically, MySQL or PostgreSQL can be used.

[0105] Specific operation of the system

[0106] 1. User Input

[0107] The user uses the device to input in text form the stress, anxiety, and specific worries they are feeling. For example, they might input, "Recently, work has been busy and I'm feeling stressed."

[0108] 2. Text Data Analysis

[0109] The device receives the input and sends it as text data to the server. The server then uses natural language processing technology to analyze the text data, tokenize it, and extract keywords. For example, keywords such as "work" and "stress" are extracted.

[0110] 3. Suggestions for relaxation methods

[0111] Based on the analysis results, the server uses these keywords to create a list of appropriate relaxation methods and suggests them to the user, such as "deep breathing," "aromatherapy," and "yoga."

[0112] 4. Search for service providers

[0113] The server searches for service providers related to the proposed relaxation method. It identifies nearby service providers from databases and online resources based on the user's location and other information. For example, it searches for information on "nearby aromatherapy salons" or "yoga classes."

[0114] 5. Presentation of results

[0115] The terminal visually presents information about the service provider to the user. For example, a text such as "Click here for the nearest aromatherapy salon" and a link are displayed on the user's screen.

[0116] 6. Receiving and Analyzing Feedback

[0117] The user tries out the provided relaxation methods and inputs their impressions and effects into the device. For example, they might input, "I tried aromatherapy and it helped me relax a little." This feedback is sent from the device to the server, which then analyzes it.

[0118] 7. Improving proposal accuracy

[0119] The server analyzes the received feedback and applies the results to the machine learning model, which improves the accuracy of relaxation method suggestions from the next time onwards.

[0120] Prompt Sentence Examples

[0121] "When a user types, 'I've been busy at work lately and feeling stressed,' create a system that suggests appropriate relaxation methods and nearby service providers. Specifically, implement a way to extract keywords using natural language processing, search for information on the suggested relaxation methods and related service providers, and visually present them to the user."

[0122] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0123] Step 1: Receiving User Input

[0124] The user inputs the stress, anxiety, and specific worries they are experiencing into the device in text format, and the device receives this input as text data.

[0125] input:

[0126] Text data entered by the user, such as "I've been busy at work lately and feeling stressed."

[0127] output:

[0128] Text data sent to the device.

[0129] Specific behavior:

[0130] The user enters "I've been busy at work lately and it's stressful" into the input field of the terminal and presses the send button.

[0131] The terminal transmits this text data to the server.

[0132] Step 2: Analyzing the text data

[0133] The server analyzes the received text data, tokenizing the input and extracting keywords using natural language processing techniques, specifically using the spaCy and NLTK libraries.

[0134] input:

[0135] Text data sent from the device: "Work has been busy lately and I'm feeling stressed."

[0136] output:

[0137] Tokenized keywords (e.g., "work" and "stress").

[0138] Specific behavior:

[0139] The server breaks down the text data into tokens and extracts important keywords.

[0140] For example, it breaks down the data into tokens such as "recently," "work," "busy," "stressed," and "piling up," and then extracts "work" and "stress" from these as keywords.

[0141] Step 3: Suggested relaxation techniques

[0142] The server then lists appropriate relaxation methods based on the extracted keywords and analysis results, and suggests these methods to the user.

[0143] input:

[0144] Extracted keywords (e.g., "work," "stress").

[0145] output:

[0146] A list of suggested relaxation techniques (e.g., "deep breathing," "aromatherapy," "yoga").

[0147] Specific behavior:

[0148] The server searches a database for relaxation methods related to the keywords and lists them in order of priority.

[0149] For example, select methods such as "deep breathing," "aromatherapy," and "yoga" and create a list.

[0150] Step 4: Find a service provider

[0151] The server searches for service providers related to the proposed relaxation method using databases and online resources, taking into account the user's location.

[0152] input:

[0153] A list of suggested relaxation techniques (e.g., "deep breathing," "aromatherapy," "yoga"), and the user's location.

[0154] output:

[0155] Information about service providers (e.g., "aromatherapy salons near me" or "yoga classes").

[0156] Specific behavior:

[0157] The server searches for nearby service providers from databases and online resources based on the user's location information.

[0158] Collect information on salons related to "aromatherapy" and classes related to "yoga."

[0159] Step 5: Presenting the results

[0160] The terminal visually presents the information of the service provider received from the server to the user.

[0161] input:

[0162] Information about the service provider sent from the server (e.g., "Here is the nearest aromatherapy salon").

[0163] output:

[0164] Service provider information displayed on the user's device.

[0165] Specific behavior:

[0166] The device displays text such as "Click here for nearby aromatherapy salons" and a link on the user's screen.

[0167] Detailed information on relaxation methods and how to access them are also displayed.

[0168] Step 6: Receive and analyze feedback

[0169] The user performs the provided relaxation method and inputs their impressions and effects into the device, which then sends the feedback to the server, which then analyzes it.

[0170] input:

[0171] User-supplied feedback (e.g., "I tried aromatherapy and it helped me relax a bit").

[0172] output:

[0173] Parsed feedback data.

[0174] Specific behavior:

[0175] The user types "I tried aromatherapy and it helped me relax a bit" into the device and presses the send button.

[0176] The terminal sends this feedback to the server.

[0177] Step 7: Improving the accuracy of the proposals

[0178] The server analyzes the received feedback and applies the results to the machine learning model, which improves the accuracy of relaxation method suggestions from the next time onwards.

[0179] input:

[0180] The feedback data received.

[0181] output:

[0182] Improved machine learning models for more accurate recommendations.

[0183] Specific behavior:

[0184] The server analyzes the feedback data and evaluates the effectiveness of the relaxation methods.

[0185] Based on the evaluation results, the machine learning model will be retrained to improve the accuracy of proposals from next time onwards.

[0186] (Application example 1)

[0187] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0188] Users of autonomous vehicles want to maintain their mental health by having opportunities to relax while traveling or waiting. However, it is difficult to suggest appropriate relaxation methods in real time according to the user's mental state and provide high-quality relaxation services based on those suggestions. It is also necessary to utilize the user's location information to accurately identify and guide nearby service providers. A system that solves these problems and makes the autonomous vehicle experience more comfortable and healthy is needed.

[0189] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0190] In this invention, the server includes means for receiving input from a user, means for analyzing the received input and identifying the user's mental state, means for suggesting an appropriate relaxation method based on the analysis result, means for searching for providers related to the suggested relaxation method, means for presenting information about the providers to the user, means for receiving and analyzing feedback from the user, means for improving the accuracy of the suggested relaxation method based on the feedback, means for providing a video guide or audio guide corresponding to the identified relaxation method, and means for using smart glasses in the autonomous vehicle to support the driver's mental health based on the specified relaxation method, thereby enabling the user to enjoy a relaxing environment in the car while receiving mental health care in real time.

[0191] Word definition

[0192] The "means for receiving input from the user" is an interface for receiving text data or voice data input by the user through the terminal.

[0193] "Means for analyzing received input content and identifying the user's mental state" refers to a function that analyzes the received user input data using natural language processing technology, etc., to identify the type of stress or anxiety the user is experiencing.

[0194] The "means for suggesting appropriate relaxation methods" is a function that selects and suggests relaxation activities suitable for the user based on the analyzed mental state.

[0195] The "means for searching for providers related to the proposed relaxation method" is a function for searching and identifying businesses or facilities that provide the proposed relaxation activity from a database.

[0196] The "means for presenting provider information to the user" is an interface that displays detailed information of the searched provider to the user visually or audibly.

[0197] The "means for receiving and analyzing feedback from users" is a function for receiving and analyzing input data such as the user's impressions and effects after trying a relaxation method.

[0198] "Means for improving the accuracy of relaxation method suggestions based on feedback" refers to a function that reflects received feedback data in the learning algorithm to improve the accuracy of suggestions from the next time onwards.

[0199] The "means for providing video guidance or audio guidance corresponding to the identified relaxation method" is a function that provides the user with video guidance or audio guidance for carrying out the proposed relaxation method.

[0200] "Means for using smart glasses in autonomous vehicles" refers to a function that uses smart glasses installed in autonomous vehicles to display and guide information on relaxation methods through the user's gaze or voice input.

[0201] MODE FOR CARRYING OUT THE INVENTION

[0202] This invention relates to a relaxation concierge system that uses smart glasses installed in autonomous vehicles to support the user's mental health. This system receives and analyzes input from the user to suggest appropriate relaxation methods, and further identifies and presents service providers based on those methods. It also has an algorithm that receives feedback after performing a relaxation method and improves its accuracy.

[0203] Hardware Configuration

[0204] Terminal: Smart glasses installed in autonomous vehicles. A device that collects and displays data through the user's gaze and voice input.

[0205] Server: A central server that analyzes data and proposes relaxation methods. Operated as a remote cloud server.

[0206] Network: A communications network for connecting terminals and servers. It uses the Internet or a dedicated communications network.

[0207] Software Configuration

[0208] Natural Language Processing Library (NLPProcessor): Analyzes user input, extracts keywords, and identifies mental states.

[0209] Location Service API (LocationServiceAPI): Obtains the user's location information and searches for nearby relaxation providers.

[0210] Relaxation Provider API (RelaxationServiceAPI): Provides relaxation method suggestions and related provider information to the server.

[0211] Processing flow

[0212] The user inputs text or voice through the smart glasses, saying, "Recently, work has been busy and I'm feeling stressed." This input is received by the device and sent to the server via the network. The server uses an NLP processor to analyze the input and extract keywords such as "work" and "stress." This identifies the user's mental state as "caused by work-related stress."

[0213] Next, the server executes a relaxation method suggestion function, suggesting techniques such as "deep breathing," "aromatherapy," and "yoga" to the user. A video guide or audio guide corresponding to the suggested relaxation method is displayed on the smart glasses. In addition, information on nearby relaxation providers is obtained via a location information service API. For example, "aromatherapy salons" and "yoga studios" near the user's current location are displayed.

[0214] The user performs the relaxation technique and then inputs feedback through the smart glasses. This feedback is then sent back to the server for analysis. The server then uses the collected feedback to train the suggestion algorithm and improve the accuracy of future suggestions.

[0215] Specific examples

[0216] User input: "Work has been busy and stressful lately."

[0217] Suggested relaxation methods: "Deep breathing," "Aromatherapy," and "Yoga"

[0218] Suggested prompt examples:

[0219] Please suggest ways to reduce the stress you feel at work.

[0220] "Tell me how to relax"

[0221] "Find nearby relaxation facilities"

[0222] This system provides users with relaxation techniques while driving or waiting, enabling them to maintain their mental health. Feedback is also used based on the results of practicing the provided relaxation techniques, improving the accuracy of the suggestions.

[0223] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0224] Detailed explanation of the processing steps

[0225] Step 1:

[0226] Input: A user types or speaks through smart glasses, "Work has been busy and stressful lately."

[0227] Action: The device receives this input and sends it over the network to the server.

[0228] Data processing and calculation: Receives data (text or voice data) input from the smart glasses and sends it to the server as digital data.

[0229] Output: The input data is sent to the server.

[0230] Step 2:

[0231] Input: User input data sent from the device.

[0232] How it works: The server uses the NLPProcessor to parse the input data it receives and extract important keywords.

[0233] Data processing and calculation: Using natural language processing, the input data is tokenized and keywords such as "work" and "stress" are extracted.

[0234] Output: Extracted keywords.

[0235] Step 3:

[0236] Input: Extracted keywords (e.g., "work" and "stress").

[0237] Operation: The server runs an algorithm to suggest relaxation methods based on the extracted keywords.

[0238] Data processing and calculation: Keywords are compared with past databases to select the most suitable relaxation method (e.g., "deep breathing," "aromatherapy," "yoga").

[0239] Output: A list of suggested relaxation methods.

[0240] Step 4:

[0241] Input: A list of suggested relaxation methods.

[0242] How it works: The server prepares video and audio guides corresponding to relaxation methods and sends them to the smart glasses.

[0243] Data processing and calculation: Searching and selecting video and audio files corresponding to the relaxation methods.

[0244] Output: Video guide and audio guide data.

[0245] Step 5:

[0246] Input: A list of suggested relaxation methods and the user's location.

[0247] How it works: The server uses the LocationService API to search for nearby relaxation providers and sends them to the smart glasses.

[0248] Data processing and calculation: Based on the user's location information, nearby facilities that offer the proposed relaxation methods are identified and information is collected.

[0249] Output: A list of nearby relaxation providers.

[0250] Step 6:

[0251] Input: Video and audio guide data, and a list of nearby relaxation providers.

[0252] How it works: The device displays this information on the smart glasses display.

[0253] Data processing and calculation: Present data to the user visually or audibly.

[0254] Output: The user receives information through the smart glasses that encourages them to practice relaxation techniques and provides guidance on nearby facilities.

[0255] Step 7:

[0256] Input: After the user performs the relaxation technique, they enter feedback about their experience.

[0257] Operation: The terminal receives the user's feedback and transmits it to the server via the network.

[0258] Data processing and calculation: Feedback data is collected and stored in a database.

[0259] Output: User feedback is sent to the server.

[0260] Step 8:

[0261] Input: User feedback.

[0262] How it works: The server analyzes the feedback and uses it to improve the accuracy of the proposed algorithm.

[0263] Data processing and calculation: Improve the accuracy of suggestions by re-training the algorithm based on feedback data.

[0264] Output: Relaxation suggestions will be performed more accurately from the next time onwards.

[0265] This series of processes allows users to receive efficient mental care while in an autonomous vehicle.

[0266] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0267] This invention provides a system that combines an emotion engine to support users' mental health care. This system analyzes input from the user and suggests appropriate relaxation methods based on the user's emotional state. It also has the function of searching for service providers related to the suggested relaxation methods and presenting them to the user.

[0268] First, the user inputs their daily stress, anxiety, and specific worries into the device. The system then incorporates an emotion engine to recognize emotions from the user's facial expressions and voice. The device then transmits the user's text input, facial expression analysis, and voice analysis data to the server.

[0269] The server analyzes the received text data using natural language processing technology to extract key keywords. It also analyzes the user's emotions recognized by the emotion engine and compares them with the mental state extracted from the text data.

[0270] For example, consider the case where a user inputs "I've been busy at work lately and feeling stressed," and emotions of tension and anxiety are recognized from their facial expressions and voice. The server extracts the keywords "work" and "stress," and by combining these with the emotions identified by the emotion engine as "tension" and "anxiety," identifies the user's mental state as "stress, tension, and anxiety due to work."

[0271] Next, the server will list appropriate relaxation methods based on the identified mental state, such as "deep breathing," "aromatherapy," and "yoga." These relaxation methods are intended to reduce the tension and anxiety identified by the emotion engine.

[0272] The server then searches for service providers related to the selected relaxation method, identifying nearby aromatherapy salons and yoga classes based on the user's location information from affiliated databases and online resources, and transmitting information about the identified service providers to the device.

[0273] The terminal visually presents the information about the service provider to the user. For example, the terminal displays a text message such as "Click here for the nearest aromatherapy salon" and a link on the user's screen.

[0274] After a user visits a salon or class and tries out a relaxation method, the device receives feedback about the effects and impressions of the experience. For example, if a user types, "I tried aromatherapy and it helped me relax a little," the device sends this feedback to the server.

[0275] Finally, the server analyzes the received feedback and reflects the analysis results in the emotion engine, which improves the accuracy of relaxation method suggestions from the next time onwards and makes suggestions that are more suited to the user's individual emotional state.

[0276] As a concrete example of this system, if a user inputs "I can't sleep at night and I'm frustrated lately," and the server recognizes fatigue from the user's facial expression and voice, it will combine keywords such as "can't sleep" and "frustrated" with "fatigue" to identify the symptom as "fatigue and frustration due to lack of sleep." In this case, the server will suggest methods such as "aromatherapy" and "relaxation music," and search for and display nearby related service providers.

[0277] In this way, by combining an emotion engine, the present invention provides a system that proposes more accurate relaxation methods based on the user's emotional state and quickly searches for and presents service providers, allowing users to easily find the relaxation method that best suits them and effectively relieve stress.

[0278] The processing flow will be explained below.

[0279] Step 1:

[0280] The user inputs information about their stress and anxiety, for example, "Work has been busy lately and I'm feeling stressed."

[0281] Step 2:

[0282] The device receives the user's input as text data, and also acquires the user's facial expression and voice data.

[0283] Step 3:

[0284] The terminal transmits the received text data, facial expression data, and voice data to the server.

[0285] Step 4:

[0286] The server receives the data sent from the terminal.

[0287] Step 5:

[0288] The server uses natural language processing technology to analyze the text data and extract key keywords. In this case, the keywords extracted are "work" and "stress."

[0289] Step 6:

[0290] The server uses an emotion engine to analyze the received facial expression and voice data and recognize the user's emotions, such as "tension" or "anxiety."

[0291] Step 7:

[0292] The server compares the keywords extracted from the text data with the emotions recognized by the emotion engine to identify the user's mental state. Specifically, it identifies "stress, tension, and anxiety due to work."

[0293] Step 8:

[0294] Based on the identified mental state, the server will then list appropriate relaxation methods, such as "deep breathing," "aromatherapy," and "yoga."

[0295] Step 9:

[0296] The server searches for service providers related to the selected relaxation method, searching a database for nearby aromatherapy salons and yoga classes based on the user's location information.

[0297] Step 10:

[0298] The server prepares information about the service providers obtained as a result of the search and transmits it to the terminal.

[0299] Step 11:

[0300] The terminal receives the information about the service provider sent from the server.

[0301] Step 12:

[0302] The terminal visually displays the received information about the service provider to the user, providing a text such as "Click here for nearby aromatherapy salons" and a link.

[0303] Step 13:

[0304] The user can then take action based on the displayed information about relaxation methods and service providers, for example, visiting a nearby aromatherapy salon.

[0305] Step 14:

[0306] After performing the relaxation method, the user inputs the results as feedback into the terminal, for example, "I tried aromatherapy and it helped me relax a little."

[0307] Step 15:

[0308] The terminal receives feedback from the user and transmits it to the server.

[0309] Step 16:

[0310] The server receives the feedback sent from the terminal.

[0311] Step 17:

[0312] The server analyzes the feedback content and evaluates the effectiveness of the relaxation method.

[0313] Step 18:

[0314] The server reflects the analysis results in the emotion engine, improving the accuracy of relaxation method suggestions from the next time onwards.

[0315] This series of steps allows users to easily find the relaxation method that best suits them and provides effective stress relief measures based on their emotions and mental state.

[0316] Example 2

[0317] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0318] In modern society, users experience a great deal of stress and anxiety due to their daily lives and work. To properly care for these mental health issues, a system is needed that can accurately assess the user's condition and suggest optimal relaxation methods. It is also necessary to analyze the user's emotional state in real time and respond quickly according to that state. Furthermore, it is necessary to immediately provide information on service providers related to the suggested relaxation methods, thereby reducing the burden on the user. Conventional systems have had difficulty meeting these requirements, so providing customized mental health care to users has become a challenge.

[0319] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0320] In this invention, the server includes means for receiving input from a user, means for analyzing the received input and identifying the user's mental state, means for suggesting an appropriate relaxation method based on the analysis results, means for searching for service providers related to the suggested relaxation method, means for presenting information about the service providers to the user, means for collecting the user's facial expression data and voice data and analyzing them using an emotion engine, means for receiving and analyzing feedback from the user, and means for improving the accuracy of suggested relaxation methods based on the feedback. This enables accurate suggestions of relaxation methods based on the user's individual mental state and rapid search and presentation of related service providers.

[0321] "User input" is information provided by a user to a terminal through text, voice, or facial expressions.

[0322] "Mental state" refers to the mental and psychological state of the user, such as stress, anxiety, and worries.

[0323] "Relaxation methods" are methods such as deep breathing, aromatherapy, and yoga that are proposed as means for improving the user's mental state.

[0324] "Service provider" refers to a person or facility that provides services related to relaxation methods.

[0325] "Suggestion accuracy" indicates the degree of accuracy and relevance when suggesting relaxation methods appropriate for the user's mental state.

[0326] An "emotion engine" is a technology or algorithm that analyzes a user's facial expressions and voice data to identify their emotional state.

[0327] "Natural language processing technology" is a technology for analyzing user input text and understanding its meaning.

[0328] "Feedback" refers to information about impressions and effects provided by users after using the proposed relaxation method or related service.

[0329] "Location information" is information that indicates the geographical location where the user currently resides.

[0330] This invention is a system that supports users' mental health care and uses technology that combines an emotion engine. Users input their daily stress, anxiety, and specific worries into a terminal, and facial expression and voice data are also collected. This input data is sent to a server and analyzed.

[0331] Hardware and software used

[0332] This system uses the following hardware and software.

[0333] Terminal: The device through which the user provides input (e.g., smartphone, PC).

[0334] Emotion engine: Technology for analyzing the user's facial expressions and voice (OpenCV, IBM Watson, Google Cloud Speech-to-Text, etc.).

[0335] Server: A computer system for analyzing and processing data.

[0336] Natural language processing technologies: Technologies for analyzing user text input (NLTK library, generative AI models such as GPT-3).

[0337] System operation explanation

[0338] 1. User Input

[0339] Users use the device to input their daily stress, anxiety, and worries in text format, while facial expression and voice data are also collected at the same time.

[0340] Example: A user types, "I've been busy at work lately and feeling stressed," and their facial expressions and voice are also recorded.

[0341] 2. Sending data to the server

[0342] The terminal transmits the input text data, facial expression data, and voice data to the server.

[0343] Technical example: A smartphone or PC sends data to a server as an HTTP request.

[0344] 3. Text Data Analysis

[0345] The server uses natural language processing technology to analyze the received text data and extract key keywords.

[0346] Technologies used: Python's NLTK library and generative AI models (such as GPT-3).

[0347] Example: Extract the keywords "work" and "stress" from the analysis results of the text "I've been busy at work lately and feeling stressed."

[0348] 4. Emotion Data Analysis

[0349] The server uses an emotion engine to analyze facial and voice data to identify the user's emotional state (e.g., tension, anxiety).

[0350] Technologies used: OpenCV for facial expression recognition, IBM Watson and Google Cloud Speech-to-Text for voice emotion recognition.

[0351] For example, "tension" can be recognized from facial expressions, and "anxiety" can be recognized from voice.

[0352] 5. Relaxation suggestions

[0353] Based on the analysis results, the server identifies the user's mental state and suggests appropriate relaxation methods (deep breathing, aromatherapy, yoga, etc.).

[0354] Example: Suggest "deep breathing," "aromatherapy," and "yoga" to combat "stress, tension, and anxiety caused by work."

[0355] 6. Search for a service provider

[0356] The server searches affiliated databases and online resources for service providers related to relaxation techniques.

[0357] The search for service providers is carried out based on the user's location information.

[0358] For example: Identify nearby aromatherapy salons or yoga classes.

[0359] 7. Presentation of Service Provider Information

[0360] The terminal visually presents the search results to the user and provides information on nearby service providers.

[0361] Example: Displaying text and a link such as "Click here for aromatherapy salons near you" on the user screen.

[0362] 8. User Feedback

[0363] After actually using the service, the user inputs the results and impressions into the terminal as feedback.

[0364] Example: User types, "I tried aromatherapy and it helped me relax a bit."

[0365] 9. Incorporating Feedback

[0366] The server analyzes the received feedback and reflects it in the emotion engine, thereby improving the accuracy of relaxation method suggestions from the next time onwards.

[0367] Technical example: Integrating feedback into the system by retraining machine learning models.

[0368] Through the above series of steps, the present invention provides a system that effectively supports users' mental health care. By combining specific technologies and operations, it is possible to accurately suggest relaxation methods based on the user's emotional state and quickly search for and present service providers.

[0369] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0370] Step 1:

[0371] User Input and Data Capture

[0372] Users use a device to input their daily stress, anxiety, and worries in text format. The system also collects facial and voice data. The input data includes text, facial images, and voice files.

[0373] Input: Text data, facial expression data, voice data

[0374] How it works: The user types, "I've been busy at work lately and feeling stressed," and their facial expressions and voice are also recorded. The device's camera and microphone capture the user's facial expressions and voice in real time.

[0375] Output: Input data (text, facial expressions, voice)

[0376] Step 2:

[0377] Sending data to the server

[0378] The terminal transmits the collected text data, facial expression data, and voice data to the server.

[0379] Input: Input data (text, facial expressions, voice)

[0380] How it works: Your smartphone or PC sends data to the server as an HTTP request, which is then formatted and encoded.

[0381] Output: Received data on the server (text, facial expressions, voice)

[0382] Step 3:

[0383] Text data analysis

[0384] The server analyzes the received text data using natural language processing technology and extracts key keywords.

[0385] Input: Received text data on the server

[0386] How it works: It uses Python's NLTK library and generative AI models (such as GPT-3) to extract important keywords from text.

[0387] Output: Extracted keywords (e.g., "work" and "stress")

[0388] Step 4:

[0389] Emotional Data Analysis

[0390] The server uses an emotion engine to analyze facial expression and voice data to identify the user's emotional state.

[0391] Input: Received facial expression data and voice data on the server

[0392] Operation: Classifies emotional states using OpenCV for facial expression recognition and IBM Watson or Google Cloud Speech-to-Text for voice emotion recognition. Example: "tension" or "anxiety."

[0393] Output: Identified emotion (e.g., "tension" or "anxiety")

[0394] Step 5:

[0395] Suggestions for relaxation methods

[0396] The server combines the analyzed keywords with the user's emotional state to identify the user's mental state and list appropriate relaxation methods.

[0397] Input: extracted keywords, identified sentiments

[0398] How it works: It detects the user's mental state and runs an algorithm that selects relaxation methods (e.g., "deep breathing," "aromatherapy," "yoga") based on that.

[0399] Output: A list of suggested relaxation techniques (e.g., "deep breathing," "aromatherapy," "yoga")

[0400] Step 6:

[0401] Search for a service provider

[0402] The server searches affiliated databases and online resources for service providers related to relaxation methods, and identifies the service provider based on the user's location information.

[0403] Input: List of relaxation methods, user location

[0404] How it works: Searches for nearby service providers (e.g., aromatherapy salons or yoga classes) through affiliated databases and APIs, and obtains rankings and reviews.

[0405] Output: List of relevant service providers with details

[0406] Step 7:

[0407] Present service provider information

[0408] The terminal visually presents information about service providers to the user based on the search results.

[0409] Input: List of relevant service providers and details

[0410] Operation: Displays information about service providers on the user interface. For example, it displays text such as "Click here for nearby aromatherapy salons," links, and map information on the screen.

[0411] Output: Service provider information presented to the user

[0412] Step 8:

[0413] User Feedback

[0414] After actually using the service, the user inputs feedback about its effectiveness and impressions into the terminal.

[0415] Input: User feedback data (text format)

[0416] What it does: Displays a feedback form and allows the user to fill it in. For example, the user might type, "I tried aromatherapy and it helped me relax a bit."

[0417] Output: Feedback data

[0418] Step 9:

[0419] Reflecting feedback

[0420] The server analyzes the received feedback and reflects it in the emotion engine, thereby improving the accuracy of relaxation method suggestions from the next time onwards.

[0421] Input: User feedback data

[0422] How it works: The feedback data is analyzed and used as data to retrain the emotion engine model, thereby improving the accuracy of the proposed algorithm.

[0423] Output: Emotion engine and proposed algorithm with improved accuracy

[0424] (Application example 2)

[0425] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0426] Conventional mental healthcare systems estimate a user's mental state based on text input and suggest relaxation methods. However, in order to recognize the user's emotional state with greater accuracy, it is necessary to incorporate facial expression and voice analysis. Furthermore, systems lack the ability to not only suggest appropriate relaxation methods but also to quickly search for and present relevant service providers. Furthermore, systems lack the ability to identify service providers based on the user's location information or to incorporate user feedback to improve the accuracy of future suggestions.

[0427] The identification process performed by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving input from the user, means for analyzing the received input and identifying the user's mental state, means for recognizing the user's emotional state through facial expression analysis and voice analysis, means for suggesting an appropriate relaxation method based on the analysis results, means for searching for service providers related to the suggested relaxation method, means for presenting information about the service providers to the user, means for receiving and analyzing feedback from the user, and means for improving the accuracy of relaxation method suggestions based on the feedback. This makes it possible to recognize the user's emotional state with higher accuracy and quickly identify and present appropriate relaxation methods and related service providers. Furthermore, using the feedback improves the accuracy of suggestions from the next time onwards.

[0428] "Means for receiving input from a user" refers to a device that allows a user to input text or voice and software for processing the input.

[0429] The "means for analyzing the received input content and identifying the user's mental state" refers to an algorithm and processing device that analyzes the text and voice data entered by the user and infers the user's emotions and psychological state from the content.

[0430] "Means for recognizing a user's emotional state through facial expression analysis and voice analysis" refers to technology that analyzes a user's facial expressions and tone of voice and recognizes their emotional state from the results.

[0431] The "means for suggesting an appropriate relaxation method based on the analysis results" refers to an algorithm and processing device for selecting and suggesting a relaxation method based on the analysis results of the user's mental and emotional state.

[0432] "Means for searching for service providers related to the proposed relaxation method" refers to a database and search engine for searching and identifying businesses and facilities that provide services according to the proposed relaxation method.

[0433] The "means for presenting information of service providers to a user" refers to a user interface and software for displaying the searched information of service providers on a user's device.

[0434] The "means for receiving and analyzing feedback from the user" refers to a means for collecting and analyzing the user's impressions and experiences after implementing the relaxation method.

[0435] The "means for improving the accuracy of relaxation method suggestions based on feedback" refers to an algorithm and processing device that uses the results of analyzing user feedback to improve the accuracy of relaxation method suggestions from the next time onwards.

[0436] The "means for identifying nearby service providers based on the user's location information" refers to a geographic information system and search engine for identifying nearby relevant service providers based on the user's current location information.

[0437] "Means for analyzing user input using natural language processing technology" refers to natural language processing algorithms and software that analyze text data entered by users and extract key keywords and emotions.

[0438] The present invention is a system for supporting a user's mental health care. This system receives input data from the user, analyzes it, and suggests appropriate relaxation methods. It also has the function of searching for service providers related to the suggested relaxation methods and presenting them to the user. The system also receives feedback and improves the accuracy of relaxation method suggestions from the next time onwards.

[0439] Hardware and Software Examples

[0440] 1. Hardware:

[0441] Smartphones (iOS and Android devices)

[0442] server

[0443] 2. Software:

[0444] Natural Language Processing: Google Cloud Natural Language API

[0445] Facial expression analysis: Amazon Rekognition

[0446] Speech analysis: Google Cloud Speech-to-Text API

[0447] Location search: Google Maps API

[0448] Data processing and calculation flow

[0449] 1. Receiving user input:

[0450] Users use a smartphone application to input their stress and anxiety levels via text or voice, while facial expression data is also collected via the camera.

[0451] 2. Analysis of input data:

[0452] The text data is analyzed using the Google Cloud Natural Language API to extract key keywords and the user's mental state.

[0453] Facial expression data is analyzed using Amazon Rekognition to identify the user's emotions.

[0454] The audio data is converted to text using the Google Cloud Speech-to-Text API, and then sentiment analysis is performed using the Google Cloud Natural Language API.

[0455] 3. Relaxation suggestions:

[0456] Based on the analysis results, a database is consulted to suggest appropriate relaxation techniques (e.g., deep breathing, aromatherapy, yoga).

[0457] 4. Search and present service providers:

[0458] Service providers related to the proposed relaxation method (e.g., aromatherapy salons, yoga classes) are searched for using the user's location information using the Google Maps API and presented to the user.

[0459] 5. Feedback Collection and Analysis:

[0460] After trying out a relaxation technique, users can enter their impressions and feedback through the application. This feedback is analyzed by the server and used to improve the accuracy of future suggestions.

[0461] Specific examples

[0462] 1. User Input:

[0463] The user types, "Work has been busy and stressful lately."

[0464] Facial expression analysis recognizes the emotion of "tension" from the user's face.

[0465] Voice analysis identifies the emotion of "anxiety" from voice.

[0466] 2. Relaxation suggestions:

[0467] The server suggests "deep breathing," "aromatherapy," and "yoga" as suitable relaxation methods.

[0468] 3. Service Provider Presentation:

[0469] The text and link "Here are the nearest aromatherapy salons" will appear on the app screen.

[0470] Prompt Sentence Examples

[0471] User input: "Work has been busy and stressful lately."

[0472] Recognized emotional state: Facial expression analysis "tension", voice analysis "anxiety"

[0473] Extracted keywords: "work", "stress"

[0474] Suggested relaxation methods: "Deep breathing", "Aromatherapy", "Yoga"

[0475] Search for nearby service providers: Google Maps API

[0476] This allows for more accurate recognition of the user's emotional state, enabling the system to suggest appropriate relaxation methods and quickly search for and present relevant service providers. Furthermore, by using user feedback, the system can improve the accuracy of suggestions from the next time onwards, providing a more personalized experience.

[0477] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0478] Step 1:

[0479] A user inputs text or voice using a smartphone application.

[0480] Input: User text input, voice input, and facial expression data

[0481] Specific behavior:

[0482] The user inputs the stress or anxiety they are experiencing through text or voice.

[0483] Facial expression data is collected via a smartphone camera.

[0484] Output: Input data (text, voice, facial expression)

[0485] Step 2:

[0486] The terminal transmits the collected input data to the server.

[0487] Input: Input data (text, voice, facial expression)

[0488] Specific behavior:

[0489] The terminal transmits the text data as is, the voice data as a voice file, and the facial expression data as an image to the server.

[0490] Output: Input data sent to the server

[0491] Step 3:

[0492] The server analyzes the received text data using the Google Cloud Natural Language API to extract key keywords and mental states.

[0493] Input: Text data

[0494] Specific behavior:

[0495] The server calls the Google Cloud Natural Language API to parse the text data.

[0496] Extract key keywords (e.g., "work," "stress") and the user's mental state.

[0497] Output: Key keywords and mental states

[0498] Step 4:

[0499] The server analyzes the facial expression data using Amazon Rekognition and recognizes the user's emotions.

[0500] Input: facial expression data (image)

[0501] Specific behavior:

[0502] The server calls Amazon Rekognition to analyze the facial expression data.

[0503] Recognize the user's emotional state (e.g., "tension") from facial expressions.

[0504] Output: Perceived emotional state

[0505] Step 5:

[0506] The server converts the voice data into text using the Google Cloud Speech-to-Text API, and then analyzes the sentiment using the Google Cloud Natural Language API.

[0507] Input: Audio data

[0508] Specific behavior:

[0509] The server calls the Google Cloud Speech-to-Text API to convert the audio data into text.

[0510] The converted text data is then analyzed again using the Google Cloud Natural Language API to identify sentiment.

[0511] Output: Perceived emotional state

[0512] Step 6:

[0513] The server refers to a database to suggest appropriate relaxation methods based on the analysis results.

[0514] Input: Primary keywords, perceived emotional state

[0515] Specific behavior:

[0516] Based on the analysis results, the most suitable relaxation method for the user (e.g., "deep breathing," "aromatherapy," "yoga") is selected from a database.

[0517] Output: Proposed relaxation method

[0518] Step 7:

[0519] The server uses the Google Maps API to search for nearby service providers related to the proposed relaxation method.

[0520] Input: Proposed relaxation method, user location

[0521] Specific behavior:

[0522] Call the Google Maps API to search for service providers based on the user's location information.

[0523] Identify service providers suitable for the proposed relaxation method.

[0524] Output: Service provider information

[0525] Step 8:

[0526] The terminal presents information about the service provider to the user.

[0527] Input: Service provider information

[0528] Specific behavior:

[0529] Display information about the service provider (e.g., "Click here for the nearest aromatherapy salon") in text and link format on the user's app screen.

[0530] Output: Information presented to the user

[0531] Step 9:

[0532] After the user tries the relaxation techniques, they enter feedback through the application.

[0533] Input: User feedback (text)

[0534] Specific behavior:

[0535] The user inputs the effects and impressions of the relaxation method in text format.

[0536] Output: Input feedback

[0537] Step 10:

[0538] The terminal transmits the feedback from the user to the server.

[0539] Input: User feedback

[0540] Specific behavior:

[0541] The terminal transmits the feedback text data to the server.

[0542] Output: Feedback sent to the server

[0543] Step 11:

[0544] The server analyzes the received feedback and updates the database to improve the accuracy of relaxation method suggestions from the next time onwards.

[0545] Input: Feedback data

[0546] Specific behavior:

[0547] The feedback data will be analyzed and the results will be used to update the database to improve the accuracy of relaxation method suggestions.

[0548] Output: Updated database

[0549] The above processing steps enable highly accurate recognition of the user's emotional state, suggesting appropriate relaxation methods, and quickly searching and presenting relevant service providers. Furthermore, by using feedback, the accuracy of suggestions can be improved in future visits, providing a more personalized experience.

[0550] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0551] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0552] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0553] [Second embodiment]

[0554] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0555] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0556] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0557] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0558] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0559] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0560] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0561] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0562] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0563] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0564] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0565] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0566] This invention provides a system that proposes appropriate relaxation methods and stress-relief activities for users with mental health concerns. This system analyzes user input, proposes relaxation methods based on the results, and searches for and presents related service providers.

[0567] First, the user inputs into the device the stress, anxiety, and specific worries they experience in their daily lives. The device then sends this input as text data to the server. The server then analyzes the received text data using natural language processing technology. Specifically, it tokenizes the input and extracts keywords.

[0568] For example, consider the case where a user enters, "Recently, I've been busy at work and feeling stressed." In this case, the server extracts keywords such as "work" and "stress" and identifies the user's mental state as "caused by stress from work."

[0569] Next, the server executes a means to suggest relaxation methods based on the identified mental state. For example, it lists several relaxation methods as candidates, such as "deep breathing," "aromatherapy," and "yoga." This list prioritizes those most relevant to the user's input.

[0570] The server then executes a search for service providers related to the proposed relaxation method, identifying nearby service providers based on the user's location information from affiliated databases and online resources, thereby providing the user with information on nearby aromatherapy salons, yoga classes, relaxation facilities, etc.

[0571] The terminal visually presents the information about the service provider to the user. For example, the terminal displays a text message such as "Click here for the nearest aromatherapy salon" and a link on the user's screen.

[0572] After a user visits a salon or class and tries out a relaxation method, the device receives feedback about the effects and impressions of the experience. For example, if a user types, "I tried aromatherapy and it helped me relax a little," the device sends this feedback to the server.

[0573] Finally, the server analyzes the received feedback and applies it to the system's learning algorithm, improving the accuracy of future relaxation suggestions and making them more tailored to the user's needs.

[0574] In this way, the present invention is a system that receives input from the user, analyzes it, and suggests appropriate relaxation methods and affiliated businesses, providing a means to easily relieve stress without being noticed by those around you.

[0575] The processing flow will be explained below.

[0576] Step 1:

[0577] The user inputs information about their stress or anxiety. For example, they might input, "Work has been busy lately and I'm feeling stressed."

[0578] Step 2:

[0579] The terminal receives input from the user as text data.

[0580] Step 3:

[0581] The terminal transmits the received text data to the server.

[0582] Step 4:

[0583] The server receives the text data sent from the terminal.

[0584] Step 5:

[0585] The server uses natural language processing technology to analyze the text data and extract key keywords. In this case, the keywords extracted are "work" and "stress."

[0586] Step 6:

[0587] The server identifies the user's mental state based on the extracted keywords. In this case, "work-related stress" is identified.

[0588] Step 7:

[0589] The server then lists appropriate relaxation methods based on the identified mental state, such as "deep breathing," "aromatherapy," and "yoga."

[0590] Step 8:

[0591] The server searches for service providers related to the selected relaxation method, searching a database for nearby aromatherapy salons and yoga classes based on the user's location information.

[0592] Step 9:

[0593] The server prepares information about the service providers obtained as a result of the search and transmits it to the terminal.

[0594] Step 10:

[0595] The terminal receives the information about the service provider sent from the server.

[0596] Step 11:

[0597] The terminal displays the received information about the service provider to the user, for example, providing a text such as "Click here for nearby aromatherapy salons" and a link.

[0598] Step 12:

[0599] The user then takes action based on the displayed information about relaxation methods and service providers, for example, visiting a nearby aromatherapy salon.

[0600] Step 13:

[0601] After performing the relaxation method, the user inputs the results into the terminal as feedback, for example, "I tried aromatherapy and it helped me relax a little."

[0602] Step 14:

[0603] The terminal receives feedback from the user and transmits it to the server.

[0604] Step 15:

[0605] The server receives the feedback sent from the terminal.

[0606] Step 16:

[0607] The server analyzes the feedback content and evaluates the effectiveness of the relaxation method.

[0608] Step 17:

[0609] The server reflects the analysis results in the system's learning algorithm, improving the accuracy of proposals from next time onwards.

[0610] Through this series of steps, users can easily find the relaxation method that best suits them and effectively relieve stress.

[0611] Example 1

[0612] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0613] In recent years, the number of people suffering from stress and anxiety has been increasing, but there is a lack of specific relaxation methods to appropriately alleviate these stresses and a means to quickly obtain information on support service providers. Furthermore, there is also a lack of mechanisms to provide feedback on the effectiveness of provided relaxation methods and improve the accuracy of suggestions for future sessions. Therefore, there is a need for a system that not only analyzes user input, suggests appropriate relaxation methods, and finds corresponding service providers, but also analyzes feedback to improve services.

[0614] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0615] In this invention, the server includes means for tokenizing the received input content, extracting keywords, and identifying the user's mental state, means for listing and proposing appropriate relaxation methods based on the extracted keywords and analysis results, and means for searching for service providers related to the proposed relaxation methods using databases and online resources. This allows users to instantly obtain relaxation methods and information on service providers that are suitable for them, and the accuracy of the suggestions is further improved through feedback analysis.

[0616] "User" refers to an individual who uses the system to provide input about stress and anxiety and receive information about suggested relaxation techniques and services.

[0617] "Input content" is text data provided by the user to the system, and mainly includes information related to stress and anxiety.

[0618] "Tokenization" is the process of using natural language processing techniques to break down input content into smaller units (tokens) and convert them into a form that is easier to analyze.

[0619] "Keywords" are important words or phrases extracted from text data that serve as clues to identifying the user's mental state and required relaxation methods.

[0620] "Relaxation techniques" refer to specific measures or activities that are used to reduce a user's stress or anxiety. Examples include deep breathing, aromatherapy, and yoga.

[0621] "Service provider" refers to the facility or professional who implements the proposed relaxation method, such as a salon or classroom.

[0622] A "database" is a collection of information in which the system stores information on relaxation methods and service providers.

[0623] "Online Resources" means data and information accessible via the Internet that may be used to locate a Service Provider.

[0624] "Visually presenting" refers to displaying information in a way that users can understand visually, and primarily includes text, links, maps, etc.

[0625] "Feedback" refers to the effects and impressions of the relaxation method the user has tried, and serves as data to improve the accuracy of suggestions from next time onwards.

[0626] A "machine learning model" is an algorithm that allows the system to learn from data and improve the accuracy of relaxation method suggestions from the next time onwards.

[0627] This invention is a system that suggests relaxation methods and stress-relieving activities to alleviate the mental health concerns of users. This system has the function of analyzing the user's input, suggesting relaxation methods based on the results, and searching for and presenting related service providers.

[0628] Hardware Configuration

[0629] The main hardware required to implement the system is as follows:

[0630] User device: A device through which a user inputs information about stress and anxiety, such as a smartphone or computer.

[0631] Server: A central processing unit for analyzing input information and suggesting relaxation methods and service providers.

[0632] Software Configuration

[0633] The main software required to implement the system is as follows:

[0634] Natural language processing libraries: Used to analyze input text data. Specific examples include spaCy and NLTK.

[0635] Machine learning model: Used to analyze feedback and improve the accuracy of relaxation method suggestions. Specifically, we use scikit-learn and TensorFlow.

[0636] Database: Stores information about service providers related to the proposed relaxation methods. Specifically, MySQL or PostgreSQL can be used.

[0637] Specific operation of the system

[0638] 1. User Input

[0639] The user uses the device to input in text form the stress, anxiety, and specific worries they are feeling. For example, they might input, "Recently, work has been busy and I'm feeling stressed."

[0640] 2. Text Data Analysis

[0641] The device receives the input and sends it as text data to the server. The server then uses natural language processing technology to analyze the text data, tokenize it, and extract keywords. For example, keywords such as "work" and "stress" are extracted.

[0642] 3. Suggestions for relaxation methods

[0643] Based on the analysis results, the server uses these keywords to create a list of appropriate relaxation methods and suggests them to the user, such as "deep breathing," "aromatherapy," and "yoga."

[0644] 4. Search for service providers

[0645] The server searches for service providers related to the proposed relaxation method. It identifies nearby service providers from databases and online resources based on the user's location and other information. For example, it searches for information on "nearby aromatherapy salons" or "yoga classes."

[0646] 5. Presentation of results

[0647] The terminal visually presents information about the service provider to the user. For example, a text such as "Click here for the nearest aromatherapy salon" and a link are displayed on the user's screen.

[0648] 6. Receiving and Analyzing Feedback

[0649] The user tries out the provided relaxation methods and inputs their impressions and effects into the device. For example, they might input, "I tried aromatherapy and it helped me relax a little." This feedback is sent from the device to the server, which then analyzes it.

[0650] 7. Improving proposal accuracy

[0651] The server analyzes the received feedback and applies the results to the machine learning model, which improves the accuracy of relaxation method suggestions from the next time onwards.

[0652] Prompt Sentence Examples

[0653] "When a user types, 'I've been busy at work lately and feeling stressed,' create a system that suggests appropriate relaxation methods and nearby service providers. Specifically, implement a way to extract keywords using natural language processing, search for information on the suggested relaxation methods and related service providers, and visually present them to the user."

[0654] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0655] Step 1: Receiving User Input

[0656] The user inputs the stress, anxiety, and specific worries they are experiencing into the device in text format, and the device receives this input as text data.

[0657] input:

[0658] Text data entered by the user, such as "I've been busy at work lately and feeling stressed."

[0659] output:

[0660] Text data sent to the device.

[0661] Specific behavior:

[0662] The user enters "I've been busy at work lately and it's stressful" into the input field of the terminal and presses the send button.

[0663] The terminal transmits this text data to the server.

[0664] Step 2: Analyzing the text data

[0665] The server analyzes the received text data, tokenizing the input and extracting keywords using natural language processing techniques, specifically using the spaCy and NLTK libraries.

[0666] input:

[0667] Text data sent from the device: "Work has been busy lately and I'm feeling stressed."

[0668] output:

[0669] Tokenized keywords (e.g., "work" and "stress").

[0670] Specific behavior:

[0671] The server breaks down the text data into tokens and extracts important keywords.

[0672] For example, it breaks down the data into tokens such as "recently," "work," "busy," "stressed," and "piling up," and then extracts "work" and "stress" from these as keywords.

[0673] Step 3: Suggested relaxation techniques

[0674] The server then lists appropriate relaxation methods based on the extracted keywords and analysis results, and suggests these methods to the user.

[0675] input:

[0676] Extracted keywords (e.g., "work," "stress").

[0677] output:

[0678] A list of suggested relaxation techniques (e.g., "deep breathing," "aromatherapy," "yoga").

[0679] Specific behavior:

[0680] The server searches a database for relaxation methods related to the keywords and lists them in order of priority.

[0681] For example, select methods such as "deep breathing," "aromatherapy," and "yoga" and create a list.

[0682] Step 4: Find a service provider

[0683] The server searches for service providers related to the proposed relaxation method using databases and online resources, taking into account the user's location.

[0684] input:

[0685] A list of suggested relaxation techniques (e.g., "deep breathing," "aromatherapy," "yoga"), and the user's location.

[0686] output:

[0687] Information about service providers (e.g., "aromatherapy salons near me" or "yoga classes").

[0688] Specific behavior:

[0689] The server searches for nearby service providers from databases and online resources based on the user's location information.

[0690] Collect information on salons related to "aromatherapy" and classes related to "yoga."

[0691] Step 5: Presenting the results

[0692] The terminal visually presents the information of the service provider received from the server to the user.

[0693] input:

[0694] Information about the service provider sent from the server (e.g., "Here is the nearest aromatherapy salon").

[0695] output:

[0696] Service provider information displayed on the user's device.

[0697] Specific behavior:

[0698] The device displays text such as "Click here for nearby aromatherapy salons" and a link on the user's screen.

[0699] Detailed information on relaxation methods and how to access them are also displayed.

[0700] Step 6: Receive and analyze feedback

[0701] The user performs the provided relaxation method and inputs their impressions and effects into the device, which then sends the feedback to the server, which then analyzes it.

[0702] input:

[0703] User-supplied feedback (e.g., "I tried aromatherapy and it helped me relax a bit").

[0704] output:

[0705] Parsed feedback data.

[0706] Specific behavior:

[0707] The user types "I tried aromatherapy and it helped me relax a bit" into the device and presses the send button.

[0708] The terminal sends this feedback to the server.

[0709] Step 7: Improving the accuracy of the proposals

[0710] The server analyzes the received feedback and applies the results to the machine learning model, which improves the accuracy of relaxation method suggestions from the next time onwards.

[0711] input:

[0712] The feedback data received.

[0713] output:

[0714] Improved machine learning models for more accurate recommendations.

[0715] Specific behavior:

[0716] The server analyzes the feedback data and evaluates the effectiveness of the relaxation methods.

[0717] Based on the evaluation results, the machine learning model will be retrained to improve the accuracy of proposals from next time onwards.

[0718] (Application example 1)

[0719] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0720] Users of autonomous vehicles want to maintain their mental health by having opportunities to relax while traveling or waiting. However, it is difficult to suggest appropriate relaxation methods in real time according to the user's mental state and provide high-quality relaxation services based on those suggestions. It is also necessary to utilize the user's location information to accurately identify and guide nearby service providers. A system that solves these problems and makes the autonomous vehicle experience more comfortable and healthy is needed.

[0721] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0722] In this invention, the server includes means for receiving input from a user, means for analyzing the received input and identifying the user's mental state, means for suggesting an appropriate relaxation method based on the analysis result, means for searching for providers related to the suggested relaxation method, means for presenting information about the providers to the user, means for receiving and analyzing feedback from the user, means for improving the accuracy of the suggested relaxation method based on the feedback, means for providing a video guide or audio guide corresponding to the identified relaxation method, and means for using smart glasses in the autonomous vehicle to support the driver's mental health based on the specified relaxation method, thereby enabling the user to enjoy a relaxing environment in the car while receiving mental health care in real time.

[0723] Word definition

[0724] The "means for receiving input from the user" is an interface for receiving text data or voice data input by the user through the terminal.

[0725] "Means for analyzing received input content and identifying the user's mental state" refers to a function that analyzes the received user input data using natural language processing technology, etc., to identify the type of stress or anxiety the user is experiencing.

[0726] The "means for suggesting appropriate relaxation methods" is a function that selects and suggests relaxation activities suitable for the user based on the analyzed mental state.

[0727] The "means for searching for providers related to the proposed relaxation method" is a function for searching and identifying businesses or facilities that provide the proposed relaxation activity from a database.

[0728] The "means for presenting provider information to the user" is an interface that displays detailed information of the searched provider to the user visually or audibly.

[0729] The "means for receiving and analyzing feedback from users" is a function for receiving and analyzing input data such as the user's impressions and effects after trying a relaxation method.

[0730] "Means for improving the accuracy of relaxation method suggestions based on feedback" refers to a function that reflects received feedback data in the learning algorithm to improve the accuracy of suggestions from the next time onwards.

[0731] The "means for providing video guidance or audio guidance corresponding to the identified relaxation method" is a function that provides the user with video guidance or audio guidance for carrying out the proposed relaxation method.

[0732] "Means for using smart glasses in autonomous vehicles" refers to a function that uses smart glasses installed in autonomous vehicles to display and guide information on relaxation methods through the user's gaze or voice input.

[0733] MODE FOR CARRYING OUT THE INVENTION

[0734] This invention relates to a relaxation concierge system that uses smart glasses installed in autonomous vehicles to support the user's mental health. This system receives and analyzes input from the user to suggest appropriate relaxation methods, and further identifies and presents service providers based on those methods. It also has an algorithm that receives feedback after performing a relaxation method and improves its accuracy.

[0735] Hardware Configuration

[0736] Terminal: Smart glasses installed in autonomous vehicles. A device that collects and displays data through the user's gaze and voice input.

[0737] Server: A central server that analyzes data and proposes relaxation methods. Operated as a remote cloud server.

[0738] Network: A communications network for connecting terminals and servers. It uses the Internet or a dedicated communications network.

[0739] Software Configuration

[0740] Natural Language Processing Library (NLPProcessor): Analyzes user input, extracts keywords, and identifies mental states.

[0741] Location Service API (LocationServiceAPI): Obtains the user's location information and searches for nearby relaxation providers.

[0742] Relaxation Provider API (RelaxationServiceAPI): Provides relaxation method suggestions and related provider information to the server.

[0743] Processing flow

[0744] The user inputs text or voice through the smart glasses, saying, "Recently, work has been busy and I'm feeling stressed." This input is received by the device and sent to the server via the network. The server uses an NLP processor to analyze the input and extract keywords such as "work" and "stress." This identifies the user's mental state as "caused by work-related stress."

[0745] Next, the server executes a relaxation method suggestion function, suggesting techniques such as "deep breathing," "aromatherapy," and "yoga" to the user. A video guide or audio guide corresponding to the suggested relaxation method is displayed on the smart glasses. In addition, information on nearby relaxation providers is obtained via a location information service API. For example, "aromatherapy salons" and "yoga studios" near the user's current location are displayed.

[0746] The user performs the relaxation technique and then inputs feedback through the smart glasses. This feedback is then sent back to the server for analysis. The server then uses the collected feedback to train the suggestion algorithm and improve the accuracy of future suggestions.

[0747] Specific examples

[0748] User input: "Work has been busy and stressful lately."

[0749] Suggested relaxation methods: "Deep breathing," "Aromatherapy," and "Yoga"

[0750] Suggested prompt examples:

[0751] Please suggest ways to reduce the stress you feel at work.

[0752] "Tell me how to relax"

[0753] "Find nearby relaxation facilities"

[0754] This system provides users with relaxation techniques while driving or waiting, enabling them to maintain their mental health. Feedback is also used based on the results of practicing the provided relaxation techniques, improving the accuracy of the suggestions.

[0755] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0756] Detailed explanation of the processing steps

[0757] Step 1:

[0758] Input: A user types or speaks through smart glasses, "Work has been busy and stressful lately."

[0759] Action: The device receives this input and sends it over the network to the server.

[0760] Data processing and calculation: Receives data (text or voice data) input from the smart glasses and sends it to the server as digital data.

[0761] Output: The input data is sent to the server.

[0762] Step 2:

[0763] Input: User input data sent from the device.

[0764] How it works: The server uses the NLPProcessor to parse the input data it receives and extract important keywords.

[0765] Data processing and calculation: Using natural language processing, the input data is tokenized and keywords such as "work" and "stress" are extracted.

[0766] Output: Extracted keywords.

[0767] Step 3:

[0768] Input: Extracted keywords (e.g., "work" and "stress").

[0769] Operation: The server runs an algorithm to suggest relaxation methods based on the extracted keywords.

[0770] Data processing and calculation: Keywords are compared with past databases to select the most suitable relaxation method (e.g., "deep breathing," "aromatherapy," "yoga").

[0771] Output: A list of suggested relaxation methods.

[0772] Step 4:

[0773] Input: A list of suggested relaxation methods.

[0774] How it works: The server prepares video and audio guides corresponding to relaxation methods and sends them to the smart glasses.

[0775] Data processing and calculation: Searching and selecting video and audio files corresponding to the relaxation methods.

[0776] Output: Video guide and audio guide data.

[0777] Step 5:

[0778] Input: A list of suggested relaxation methods and the user's location.

[0779] How it works: The server uses the LocationService API to search for nearby relaxation providers and sends them to the smart glasses.

[0780] Data processing and calculation: Based on the user's location information, nearby facilities that offer the proposed relaxation methods are identified and information is collected.

[0781] Output: A list of nearby relaxation providers.

[0782] Step 6:

[0783] Input: Video and audio guide data, and a list of nearby relaxation providers.

[0784] How it works: The device displays this information on the smart glasses display.

[0785] Data processing and calculation: Present data to the user visually or audibly.

[0786] Output: The user receives information through the smart glasses that encourages them to practice relaxation techniques and provides guidance on nearby facilities.

[0787] Step 7:

[0788] Input: After the user performs the relaxation technique, they enter feedback about their experience.

[0789] Operation: The terminal receives the user's feedback and transmits it to the server via the network.

[0790] Data processing and calculation: Feedback data is collected and stored in a database.

[0791] Output: User feedback is sent to the server.

[0792] Step 8:

[0793] Input: User feedback.

[0794] How it works: The server analyzes the feedback and uses it to improve the accuracy of the proposed algorithm.

[0795] Data processing and calculation: Improve the accuracy of suggestions by re-training the algorithm based on feedback data.

[0796] Output: Relaxation suggestions will be performed more accurately from the next time onwards.

[0797] This series of processes allows users to receive efficient mental care while in an autonomous vehicle.

[0798] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0799] This invention provides a system that combines an emotion engine to support users' mental health care. This system analyzes input from the user and suggests appropriate relaxation methods based on the user's emotional state. It also has the function of searching for service providers related to the suggested relaxation methods and presenting them to the user.

[0800] First, the user inputs their daily stress, anxiety, and specific worries into the device. The system then incorporates an emotion engine to recognize emotions from the user's facial expressions and voice. The device then transmits the user's text input, facial expression analysis, and voice analysis data to the server.

[0801] The server analyzes the received text data using natural language processing technology to extract key keywords. It also analyzes the user's emotions recognized by the emotion engine and compares them with the mental state extracted from the text data.

[0802] For example, consider the case where a user inputs "I've been busy at work lately and feeling stressed," and emotions of tension and anxiety are recognized from their facial expressions and voice. The server extracts the keywords "work" and "stress," and by combining these with the emotions identified by the emotion engine as "tension" and "anxiety," identifies the user's mental state as "stress, tension, and anxiety due to work."

[0803] Next, the server will list appropriate relaxation methods based on the identified mental state, such as "deep breathing," "aromatherapy," and "yoga." These relaxation methods are intended to reduce the tension and anxiety identified by the emotion engine.

[0804] The server then searches for service providers related to the selected relaxation method, identifying nearby aromatherapy salons and yoga classes based on the user's location information from affiliated databases and online resources, and transmitting information about the identified service providers to the device.

[0805] The terminal visually presents the information about the service provider to the user. For example, the terminal displays a text message such as "Click here for the nearest aromatherapy salon" and a link on the user's screen.

[0806] After a user visits a salon or class and tries out a relaxation method, the device receives feedback about the effects and impressions of the experience. For example, if a user types, "I tried aromatherapy and it helped me relax a little," the device sends this feedback to the server.

[0807] Finally, the server analyzes the received feedback and reflects the analysis results in the emotion engine, which improves the accuracy of relaxation method suggestions from the next time onwards and makes suggestions that are more suited to the user's individual emotional state.

[0808] As a concrete example of this system, if a user inputs "I can't sleep at night and I'm frustrated lately," and the server recognizes fatigue from the user's facial expression and voice, it will combine keywords such as "can't sleep" and "frustrated" with "fatigue" to identify the symptom as "fatigue and frustration due to lack of sleep." In this case, the server will suggest methods such as "aromatherapy" and "relaxation music," and search for and display nearby related service providers.

[0809] In this way, by combining an emotion engine, the present invention provides a system that proposes more accurate relaxation methods based on the user's emotional state and quickly searches for and presents service providers, allowing users to easily find the relaxation method that best suits them and effectively relieve stress.

[0810] The processing flow will be explained below.

[0811] Step 1:

[0812] The user inputs information about their stress and anxiety, for example, "Work has been busy lately and I'm feeling stressed."

[0813] Step 2:

[0814] The device receives the user's input as text data, and also acquires the user's facial expression and voice data.

[0815] Step 3:

[0816] The terminal transmits the received text data, facial expression data, and voice data to the server.

[0817] Step 4:

[0818] The server receives the data sent from the terminal.

[0819] Step 5:

[0820] The server uses natural language processing technology to analyze the text data and extract key keywords. In this case, the keywords extracted are "work" and "stress."

[0821] Step 6:

[0822] The server uses an emotion engine to analyze the received facial expression and voice data and recognize the user's emotions, such as "tension" or "anxiety."

[0823] Step 7:

[0824] The server compares the keywords extracted from the text data with the emotions recognized by the emotion engine to identify the user's mental state. Specifically, it identifies "stress, tension, and anxiety due to work."

[0825] Step 8:

[0826] Based on the identified mental state, the server will then list appropriate relaxation methods, such as "deep breathing," "aromatherapy," and "yoga."

[0827] Step 9:

[0828] The server searches for service providers related to the selected relaxation method, searching a database for nearby aromatherapy salons and yoga classes based on the user's location information.

[0829] Step 10:

[0830] The server prepares information about the service providers obtained as a result of the search and transmits it to the terminal.

[0831] Step 11:

[0832] The terminal receives the information about the service provider sent from the server.

[0833] Step 12:

[0834] The terminal visually displays the received information about the service provider to the user, providing a text such as "Click here for nearby aromatherapy salons" and a link.

[0835] Step 13:

[0836] The user can then take action based on the displayed information about relaxation methods and service providers, for example, visiting a nearby aromatherapy salon.

[0837] Step 14:

[0838] After performing the relaxation method, the user inputs the results as feedback into the terminal, for example, "I tried aromatherapy and it helped me relax a little."

[0839] Step 15:

[0840] The terminal receives feedback from the user and transmits it to the server.

[0841] Step 16:

[0842] The server receives the feedback sent from the terminal.

[0843] Step 17:

[0844] The server analyzes the feedback content and evaluates the effectiveness of the relaxation method.

[0845] Step 18:

[0846] The server reflects the analysis results in the emotion engine, improving the accuracy of relaxation method suggestions from the next time onwards.

[0847] This series of steps allows users to easily find the relaxation method that best suits them and provides effective stress relief measures based on their emotions and mental state.

[0848] Example 2

[0849] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0850] In modern society, users experience a great deal of stress and anxiety due to their daily lives and work. To properly care for these mental health issues, a system is needed that can accurately assess the user's condition and suggest optimal relaxation methods. It is also necessary to analyze the user's emotional state in real time and respond quickly according to that state. Furthermore, it is necessary to immediately provide information on service providers related to the suggested relaxation methods, thereby reducing the burden on the user. Conventional systems have had difficulty meeting these requirements, so providing customized mental health care to users has become a challenge.

[0851] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0852] In this invention, the server includes means for receiving input from a user, means for analyzing the received input and identifying the user's mental state, means for suggesting an appropriate relaxation method based on the analysis results, means for searching for service providers related to the suggested relaxation method, means for presenting information about the service providers to the user, means for collecting the user's facial expression data and voice data and analyzing them using an emotion engine, means for receiving and analyzing feedback from the user, and means for improving the accuracy of suggested relaxation methods based on the feedback. This enables accurate suggestions of relaxation methods based on the user's individual mental state and rapid search and presentation of related service providers.

[0853] "User input" is information provided by a user to a terminal through text, voice, or facial expressions.

[0854] "Mental state" refers to the mental and psychological state of the user, such as stress, anxiety, and worries.

[0855] "Relaxation methods" are methods such as deep breathing, aromatherapy, and yoga that are proposed as means for improving the user's mental state.

[0856] "Service provider" refers to a person or facility that provides services related to relaxation methods.

[0857] "Suggestion accuracy" indicates the degree of accuracy and relevance when suggesting relaxation methods appropriate for the user's mental state.

[0858] An "emotion engine" is a technology or algorithm that analyzes a user's facial expressions and voice data to identify their emotional state.

[0859] "Natural language processing technology" is a technology for analyzing user input text and understanding its meaning.

[0860] "Feedback" refers to information about impressions and effects provided by users after using the proposed relaxation method or related service.

[0861] "Location information" is information that indicates the geographical location where the user currently resides.

[0862] This invention is a system that supports users' mental health care and uses technology that combines an emotion engine. Users input their daily stress, anxiety, and specific worries into a terminal, and facial expression and voice data are also collected. This input data is sent to a server and analyzed.

[0863] Hardware and software used

[0864] This system uses the following hardware and software.

[0865] Terminal: The device through which the user provides input (e.g., smartphone, PC).

[0866] Emotion engine: Technology for analyzing the user's facial expressions and voice (OpenCV, IBM Watson, Google Cloud Speech-to-Text, etc.).

[0867] Server: A computer system for analyzing and processing data.

[0868] Natural language processing technologies: Technologies for analyzing user text input (NLTK library, generative AI models such as GPT-3).

[0869] System operation explanation

[0870] 1. User Input

[0871] Users use the device to input their daily stress, anxiety, and worries in text format, while facial expression and voice data are also collected at the same time.

[0872] Example: A user types, "I've been busy at work lately and feeling stressed," and their facial expressions and voice are also recorded.

[0873] 2. Sending data to the server

[0874] The terminal transmits the input text data, facial expression data, and voice data to the server.

[0875] Technical example: A smartphone or PC sends data to a server as an HTTP request.

[0876] 3. Text Data Analysis

[0877] The server uses natural language processing technology to analyze the received text data and extract key keywords.

[0878] Technologies used: Python's NLTK library and generative AI models (such as GPT-3).

[0879] Example: Extract the keywords "work" and "stress" from the analysis results of the text "I've been busy at work lately and feeling stressed."

[0880] 4. Emotion Data Analysis

[0881] The server uses an emotion engine to analyze facial and voice data to identify the user's emotional state (e.g., tension, anxiety).

[0882] Technologies used: OpenCV for facial expression recognition, IBM Watson and Google Cloud Speech-to-Text for voice emotion recognition.

[0883] For example, "tension" can be recognized from facial expressions, and "anxiety" can be recognized from voice.

[0884] 5. Relaxation suggestions

[0885] Based on the analysis results, the server identifies the user's mental state and suggests appropriate relaxation methods (deep breathing, aromatherapy, yoga, etc.).

[0886] Example: Suggest "deep breathing," "aromatherapy," and "yoga" to combat "stress, tension, and anxiety caused by work."

[0887] 6. Search for a service provider

[0888] The server searches affiliated databases and online resources for service providers related to relaxation techniques.

[0889] The search for service providers is carried out based on the user's location information.

[0890] For example: Identify nearby aromatherapy salons or yoga classes.

[0891] 7. Presentation of Service Provider Information

[0892] The terminal visually presents the search results to the user and provides information on nearby service providers.

[0893] Example: Displaying text and a link such as "Click here for aromatherapy salons near you" on the user screen.

[0894] 8. User Feedback

[0895] After actually using the service, the user inputs the results and impressions into the terminal as feedback.

[0896] Example: User types, "I tried aromatherapy and it helped me relax a bit."

[0897] 9. Incorporating Feedback

[0898] The server analyzes the received feedback and reflects it in the emotion engine, thereby improving the accuracy of relaxation method suggestions from the next time onwards.

[0899] Technical example: Integrating feedback into the system by retraining machine learning models.

[0900] Through the above series of steps, the present invention provides a system that effectively supports users' mental health care. By combining specific technologies and operations, it is possible to accurately suggest relaxation methods based on the user's emotional state and quickly search for and present service providers.

[0901] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0902] Step 1:

[0903] User Input and Data Capture

[0904] Users use a device to input their daily stress, anxiety, and worries in text format. The system also collects facial and voice data. The input data includes text, facial images, and voice files.

[0905] Input: Text data, facial expression data, voice data

[0906] How it works: The user types, "I've been busy at work lately and feeling stressed," and their facial expressions and voice are also recorded. The device's camera and microphone capture the user's facial expressions and voice in real time.

[0907] Output: Input data (text, facial expressions, voice)

[0908] Step 2:

[0909] Sending data to the server

[0910] The terminal transmits the collected text data, facial expression data, and voice data to the server.

[0911] Input: Input data (text, facial expressions, voice)

[0912] How it works: Your smartphone or PC sends data to the server as an HTTP request, which is then formatted and encoded.

[0913] Output: Received data on the server (text, facial expressions, voice)

[0914] Step 3:

[0915] Text data analysis

[0916] The server analyzes the received text data using natural language processing technology and extracts key keywords.

[0917] Input: Received text data on the server

[0918] How it works: It uses Python's NLTK library and generative AI models (such as GPT-3) to extract important keywords from text.

[0919] Output: Extracted keywords (e.g., "work" and "stress")

[0920] Step 4:

[0921] Emotional Data Analysis

[0922] The server uses an emotion engine to analyze facial expression and voice data to identify the user's emotional state.

[0923] Input: Received facial expression data and voice data on the server

[0924] Operation: Classifies emotional states using OpenCV for facial expression recognition and IBM Watson or Google Cloud Speech-to-Text for voice emotion recognition. Example: "tension" or "anxiety."

[0925] Output: Identified emotion (e.g., "tension" or "anxiety")

[0926] Step 5:

[0927] Suggestions for relaxation methods

[0928] The server combines the analyzed keywords with the user's emotional state to identify the user's mental state and list appropriate relaxation methods.

[0929] Input: extracted keywords, identified sentiments

[0930] How it works: It detects the user's mental state and runs an algorithm that selects relaxation methods (e.g., "deep breathing," "aromatherapy," "yoga") based on that.

[0931] Output: A list of suggested relaxation techniques (e.g., "deep breathing," "aromatherapy," "yoga")

[0932] Step 6:

[0933] Search for a service provider

[0934] The server searches affiliated databases and online resources for service providers related to relaxation methods, and identifies the service provider based on the user's location information.

[0935] Input: List of relaxation methods, user location

[0936] How it works: Searches for nearby service providers (e.g., aromatherapy salons or yoga classes) through affiliated databases and APIs, and obtains rankings and reviews.

[0937] Output: List of relevant service providers with details

[0938] Step 7:

[0939] Present service provider information

[0940] The terminal visually presents information about service providers to the user based on the search results.

[0941] Input: List of relevant service providers and details

[0942] Operation: Displays information about service providers on the user interface. For example, it displays text such as "Click here for nearby aromatherapy salons," links, and map information on the screen.

[0943] Output: Service provider information presented to the user

[0944] Step 8:

[0945] User Feedback

[0946] After actually using the service, the user inputs feedback about its effectiveness and impressions into the terminal.

[0947] Input: User feedback data (text format)

[0948] What it does: Displays a feedback form and allows the user to fill it in. For example, the user might type, "I tried aromatherapy and it helped me relax a bit."

[0949] Output: Feedback data

[0950] Step 9:

[0951] Reflecting feedback

[0952] The server analyzes the received feedback and reflects it in the emotion engine, thereby improving the accuracy of relaxation method suggestions from the next time onwards.

[0953] Input: User feedback data

[0954] How it works: The feedback data is analyzed and used as data to retrain the emotion engine model, thereby improving the accuracy of the proposed algorithm.

[0955] Output: Emotion engine and proposed algorithm with improved accuracy

[0956] (Application example 2)

[0957] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0958] Conventional mental healthcare systems estimate a user's mental state based on text input and suggest relaxation methods. However, in order to recognize the user's emotional state with greater accuracy, it is necessary to incorporate facial expression and voice analysis. Furthermore, systems lack the ability to not only suggest appropriate relaxation methods but also to quickly search for and present relevant service providers. Furthermore, systems lack the ability to identify service providers based on the user's location information or to incorporate user feedback to improve the accuracy of future suggestions.

[0959] The identification process performed by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving input from the user, means for analyzing the received input and identifying the user's mental state, means for recognizing the user's emotional state through facial expression analysis and voice analysis, means for suggesting an appropriate relaxation method based on the analysis results, means for searching for service providers related to the suggested relaxation method, means for presenting information about the service providers to the user, means for receiving and analyzing feedback from the user, and means for improving the accuracy of relaxation method suggestions based on the feedback. This makes it possible to recognize the user's emotional state with higher accuracy and quickly identify and present appropriate relaxation methods and related service providers. Furthermore, using the feedback improves the accuracy of suggestions from the next time onwards.

[0960] "Means for receiving input from a user" refers to a device that allows a user to input text or voice and software for processing the input.

[0961] The "means for analyzing the received input content and identifying the user's mental state" refers to an algorithm and processing device that analyzes the text and voice data entered by the user and infers the user's emotions and psychological state from the content.

[0962] "Means for recognizing a user's emotional state through facial expression analysis and voice analysis" refers to technology that analyzes a user's facial expressions and tone of voice and recognizes their emotional state from the results.

[0963] The "means for suggesting an appropriate relaxation method based on the analysis results" refers to an algorithm and processing device for selecting and suggesting a relaxation method based on the analysis results of the user's mental and emotional state.

[0964] "Means for searching for service providers related to the proposed relaxation method" refers to a database and search engine for searching and identifying businesses and facilities that provide services according to the proposed relaxation method.

[0965] The "means for presenting information of service providers to a user" refers to a user interface and software for displaying the searched information of service providers on a user's device.

[0966] The "means for receiving and analyzing feedback from the user" refers to a means for collecting and analyzing the user's impressions and experiences after implementing the relaxation method.

[0967] The "means for improving the accuracy of relaxation method suggestions based on feedback" refers to an algorithm and processing device that uses the results of analyzing user feedback to improve the accuracy of relaxation method suggestions from the next time onwards.

[0968] The "means for identifying nearby service providers based on the user's location information" refers to a geographic information system and search engine for identifying nearby relevant service providers based on the user's current location information.

[0969] "Means for analyzing user input using natural language processing technology" refers to natural language processing algorithms and software that analyze text data entered by users and extract key keywords and emotions.

[0970] The present invention is a system for supporting a user's mental health care. This system receives input data from the user, analyzes it, and suggests appropriate relaxation methods. It also has the function of searching for service providers related to the suggested relaxation methods and presenting them to the user. The system also receives feedback and improves the accuracy of relaxation method suggestions from the next time onwards.

[0971] Hardware and Software Examples

[0972] 1. Hardware:

[0973] Smartphones (iOS and Android devices)

[0974] server

[0975] 2. Software:

[0976] Natural Language Processing: Google Cloud Natural Language API

[0977] Facial expression analysis: Amazon Rekognition

[0978] Speech analysis: Google Cloud Speech-to-Text API

[0979] Location search: Google Maps API

[0980] Data processing and calculation flow

[0981] 1. Receiving user input:

[0982] Users use a smartphone application to input their stress and anxiety levels via text or voice, while facial expression data is also collected via the camera.

[0983] 2. Analysis of input data:

[0984] The text data is analyzed using the Google Cloud Natural Language API to extract key keywords and the user's mental state.

[0985] Facial expression data is analyzed using Amazon Rekognition to identify the user's emotions.

[0986] The audio data is converted to text using the Google Cloud Speech-to-Text API, and then sentiment analysis is performed using the Google Cloud Natural Language API.

[0987] 3. Relaxation suggestions:

[0988] Based on the analysis results, a database is consulted to suggest appropriate relaxation techniques (e.g., deep breathing, aromatherapy, yoga).

[0989] 4. Search and present service providers:

[0990] Service providers related to the proposed relaxation method (e.g., aromatherapy salons, yoga classes) are searched for using the user's location information using the Google Maps API and presented to the user.

[0991] 5. Feedback Collection and Analysis:

[0992] After trying out a relaxation technique, users can enter their impressions and feedback through the application. This feedback is analyzed by the server and used to improve the accuracy of future suggestions.

[0993] Specific examples

[0994] 1. User Input:

[0995] The user types, "Work has been busy and stressful lately."

[0996] Facial expression analysis recognizes the emotion of "tension" from the user's face.

[0997] Voice analysis identifies the emotion of "anxiety" from voice.

[0998] 2. Relaxation suggestions:

[0999] The server suggests "deep breathing," "aromatherapy," and "yoga" as suitable relaxation methods.

[1000] 3. Service Provider Presentation:

[1001] The text and link "Here are the nearest aromatherapy salons" will appear on the app screen.

[1002] Prompt Sentence Examples

[1003] User input: "Work has been busy and stressful lately."

[1004] Recognized emotional state: Facial expression analysis "tension", voice analysis "anxiety"

[1005] Extracted keywords: "work", "stress"

[1006] Suggested relaxation methods: "Deep breathing", "Aromatherapy", "Yoga"

[1007] Search for nearby service providers: Google Maps API

[1008] This allows for more accurate recognition of the user's emotional state, enabling the system to suggest appropriate relaxation methods and quickly search for and present relevant service providers. Furthermore, by using user feedback, the system can improve the accuracy of suggestions from the next time onwards, providing a more personalized experience.

[1009] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1010] Step 1:

[1011] A user inputs text or voice using a smartphone application.

[1012] Input: User text input, voice input, and facial expression data

[1013] Specific behavior:

[1014] The user inputs the stress or anxiety they are experiencing through text or voice.

[1015] Facial expression data is collected via a smartphone camera.

[1016] Output: Input data (text, voice, facial expression)

[1017] Step 2:

[1018] The terminal transmits the collected input data to the server.

[1019] Input: Input data (text, voice, facial expression)

[1020] Specific behavior:

[1021] The terminal transmits the text data as is, the voice data as a voice file, and the facial expression data as an image to the server.

[1022] Output: Input data sent to the server

[1023] Step 3:

[1024] The server analyzes the received text data using the Google Cloud Natural Language API to extract key keywords and mental states.

[1025] Input: Text data

[1026] Specific behavior:

[1027] The server calls the Google Cloud Natural Language API to parse the text data.

[1028] Extract key keywords (e.g., "work," "stress") and the user's mental state.

[1029] Output: Key keywords and mental states

[1030] Step 4:

[1031] The server analyzes the facial expression data using Amazon Rekognition and recognizes the user's emotions.

[1032] Input: facial expression data (image)

[1033] Specific behavior:

[1034] The server calls Amazon Rekognition to analyze the facial expression data.

[1035] Recognize the user's emotional state (e.g., "tension") from facial expressions.

[1036] Output: Perceived emotional state

[1037] Step 5:

[1038] The server converts the voice data into text using the Google Cloud Speech-to-Text API, and then analyzes the sentiment using the Google Cloud Natural Language API.

[1039] Input: Audio data

[1040] Specific behavior:

[1041] The server calls the Google Cloud Speech-to-Text API to convert the audio data into text.

[1042] The converted text data is then analyzed again using the Google Cloud Natural Language API to identify sentiment.

[1043] Output: Perceived emotional state

[1044] Step 6:

[1045] The server refers to a database to suggest appropriate relaxation methods based on the analysis results.

[1046] Input: Primary keywords, perceived emotional state

[1047] Specific behavior:

[1048] Based on the analysis results, the most suitable relaxation method for the user (e.g., "deep breathing," "aromatherapy," "yoga") is selected from a database.

[1049] Output: Proposed relaxation method

[1050] Step 7:

[1051] The server uses the Google Maps API to search for nearby service providers related to the proposed relaxation method.

[1052] Input: Proposed relaxation method, user location

[1053] Specific behavior:

[1054] Call the Google Maps API to search for service providers based on the user's location information.

[1055] Identify service providers suitable for the proposed relaxation method.

[1056] Output: Service provider information

[1057] Step 8:

[1058] The terminal presents information about the service provider to the user.

[1059] Input: Service provider information

[1060] Specific behavior:

[1061] Display information about the service provider (e.g., "Click here for the nearest aromatherapy salon") in text and link format on the user's app screen.

[1062] Output: Information presented to the user

[1063] Step 9:

[1064] After the user tries the relaxation techniques, they enter feedback through the application.

[1065] Input: User feedback (text)

[1066] Specific behavior:

[1067] The user inputs the effects and impressions of the relaxation method in text format.

[1068] Output: Input feedback

[1069] Step 10:

[1070] The terminal transmits the feedback from the user to the server.

[1071] Input: User feedback

[1072] Specific behavior:

[1073] The terminal transmits the feedback text data to the server.

[1074] Output: Feedback sent to the server

[1075] Step 11:

[1076] The server analyzes the received feedback and updates the database to improve the accuracy of relaxation method suggestions from the next time onwards.

[1077] Input: Feedback data

[1078] Specific behavior:

[1079] The feedback data will be analyzed and the results will be used to update the database to improve the accuracy of relaxation method suggestions.

[1080] Output: Updated database

[1081] The above processing steps enable highly accurate recognition of the user's emotional state, suggesting appropriate relaxation methods, and quickly searching and presenting relevant service providers. Furthermore, by using feedback, the accuracy of suggestions can be improved in future visits, providing a more personalized experience.

[1082] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1083] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1084] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1085] [Third embodiment]

[1086] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1087] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1088] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1089] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1090] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1091] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1092] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1093] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1094] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1095] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1096] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1097] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1098] This invention provides a system that proposes appropriate relaxation methods and stress-relief activities for users with mental health concerns. This system analyzes user input, proposes relaxation methods based on the results, and searches for and presents related service providers.

[1099] First, the user inputs into the device the stress, anxiety, and specific worries they experience in their daily lives. The device then sends this input as text data to the server. The server then analyzes the received text data using natural language processing technology. Specifically, it tokenizes the input and extracts keywords.

[1100] For example, consider the case where a user enters, "Recently, I've been busy at work and feeling stressed." In this case, the server extracts keywords such as "work" and "stress" and identifies the user's mental state as "caused by stress from work."

[1101] Next, the server executes a means to suggest relaxation methods based on the identified mental state. For example, it lists several relaxation methods as candidates, such as "deep breathing," "aromatherapy," and "yoga." This list prioritizes those most relevant to the user's input.

[1102] The server then executes a search for service providers related to the proposed relaxation method, identifying nearby service providers based on the user's location information from affiliated databases and online resources, thereby providing the user with information on nearby aromatherapy salons, yoga classes, relaxation facilities, etc.

[1103] The terminal visually presents the information about the service provider to the user. For example, the terminal displays a text message such as "Click here for the nearest aromatherapy salon" and a link on the user's screen.

[1104] After a user visits a salon or class and tries out a relaxation method, the device receives feedback about the effects and impressions of the experience. For example, if a user types, "I tried aromatherapy and it helped me relax a little," the device sends this feedback to the server.

[1105] Finally, the server analyzes the received feedback and applies it to the system's learning algorithm, improving the accuracy of future relaxation suggestions and making them more tailored to the user's needs.

[1106] In this way, the present invention is a system that receives input from the user, analyzes it, and suggests appropriate relaxation methods and affiliated businesses, providing a means to easily relieve stress without being noticed by those around you.

[1107] The processing flow will be explained below.

[1108] Step 1:

[1109] The user inputs information about their stress or anxiety. For example, they might input, "Work has been busy lately and I'm feeling stressed."

[1110] Step 2:

[1111] The terminal receives input from the user as text data.

[1112] Step 3:

[1113] The terminal transmits the received text data to the server.

[1114] Step 4:

[1115] The server receives the text data sent from the terminal.

[1116] Step 5:

[1117] The server uses natural language processing technology to analyze the text data and extract key keywords. In this case, the keywords extracted are "work" and "stress."

[1118] Step 6:

[1119] The server identifies the user's mental state based on the extracted keywords. In this case, "work-related stress" is identified.

[1120] Step 7:

[1121] The server then lists appropriate relaxation methods based on the identified mental state, such as "deep breathing," "aromatherapy," and "yoga."

[1122] Step 8:

[1123] The server searches for service providers related to the selected relaxation method, searching a database for nearby aromatherapy salons and yoga classes based on the user's location information.

[1124] Step 9:

[1125] The server prepares information about the service providers obtained as a result of the search and transmits it to the terminal.

[1126] Step 10:

[1127] The terminal receives the information about the service provider sent from the server.

[1128] Step 11:

[1129] The terminal displays the received information about the service provider to the user, for example, providing a text such as "Click here for nearby aromatherapy salons" and a link.

[1130] Step 12:

[1131] The user then takes action based on the displayed information about relaxation methods and service providers, for example, visiting a nearby aromatherapy salon.

[1132] Step 13:

[1133] After performing the relaxation method, the user inputs the results into the terminal as feedback, for example, "I tried aromatherapy and it helped me relax a little."

[1134] Step 14:

[1135] The terminal receives feedback from the user and transmits it to the server.

[1136] Step 15:

[1137] The server receives the feedback sent from the terminal.

[1138] Step 16:

[1139] The server analyzes the feedback content and evaluates the effectiveness of the relaxation method.

[1140] Step 17:

[1141] The server reflects the analysis results in the system's learning algorithm, improving the accuracy of proposals from next time onwards.

[1142] Through this series of steps, users can easily find the relaxation method that best suits them and effectively relieve stress.

[1143] Example 1

[1144] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1145] In recent years, the number of people suffering from stress and anxiety has been increasing, but there is a lack of specific relaxation methods to appropriately alleviate these stresses and a means to quickly obtain information on support service providers. Furthermore, there is also a lack of mechanisms to provide feedback on the effectiveness of provided relaxation methods and improve the accuracy of suggestions for future sessions. Therefore, there is a need for a system that not only analyzes user input, suggests appropriate relaxation methods, and finds corresponding service providers, but also analyzes feedback to improve services.

[1146] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1147] In this invention, the server includes means for tokenizing the received input content, extracting keywords, and identifying the user's mental state, means for listing and proposing appropriate relaxation methods based on the extracted keywords and analysis results, and means for searching for service providers related to the proposed relaxation methods using databases and online resources. This allows users to instantly obtain relaxation methods and information on service providers that are suitable for them, and the accuracy of the suggestions is further improved through feedback analysis.

[1148] "User" refers to an individual who uses the system to provide input about stress and anxiety and receive information about suggested relaxation techniques and services.

[1149] "Input content" is text data provided by the user to the system, and mainly includes information related to stress and anxiety.

[1150] "Tokenization" is the process of using natural language processing techniques to break down input content into smaller units (tokens) and convert them into a form that is easier to analyze.

[1151] "Keywords" are important words or phrases extracted from text data that serve as clues to identifying the user's mental state and required relaxation methods.

[1152] "Relaxation techniques" refer to specific measures or activities that are used to reduce a user's stress or anxiety. Examples include deep breathing, aromatherapy, and yoga.

[1153] "Service provider" refers to the facility or professional who implements the proposed relaxation method, such as a salon or classroom.

[1154] A "database" is a collection of information in which the system stores information on relaxation methods and service providers.

[1155] "Online Resources" means data and information accessible via the Internet that may be used to locate a Service Provider.

[1156] "Visually presenting" refers to displaying information in a way that users can understand visually, and primarily includes text, links, maps, etc.

[1157] "Feedback" refers to the effects and impressions of the relaxation method the user has tried, and serves as data to improve the accuracy of suggestions from next time onwards.

[1158] A "machine learning model" is an algorithm that allows the system to learn from data and improve the accuracy of relaxation method suggestions from the next time onwards.

[1159] This invention is a system that suggests relaxation methods and stress-relieving activities to alleviate the mental health concerns of users. This system has the function of analyzing the user's input, suggesting relaxation methods based on the results, and searching for and presenting related service providers.

[1160] Hardware Configuration

[1161] The main hardware required to implement the system is as follows:

[1162] User device: A device through which a user inputs information about stress and anxiety, such as a smartphone or computer.

[1163] Server: A central processing unit for analyzing input information and suggesting relaxation methods and service providers.

[1164] Software Configuration

[1165] The main software required to implement the system is as follows:

[1166] Natural language processing libraries: Used to analyze input text data. Specific examples include spaCy and NLTK.

[1167] Machine learning model: Used to analyze feedback and improve the accuracy of relaxation method suggestions. Specifically, we use scikit-learn and TensorFlow.

[1168] Database: Stores information about service providers related to the proposed relaxation methods. Specifically, MySQL or PostgreSQL can be used.

[1169] Specific operation of the system

[1170] 1. User Input

[1171] The user uses the device to input in text form the stress, anxiety, and specific worries they are feeling. For example, they might input, "Recently, work has been busy and I'm feeling stressed."

[1172] 2. Text Data Analysis

[1173] The device receives the input and sends it as text data to the server. The server then uses natural language processing technology to analyze the text data, tokenize it, and extract keywords. For example, keywords such as "work" and "stress" are extracted.

[1174] 3. Suggestions for relaxation methods

[1175] Based on the analysis results, the server uses these keywords to create a list of appropriate relaxation methods and suggests them to the user, such as "deep breathing," "aromatherapy," and "yoga."

[1176] 4. Search for service providers

[1177] The server searches for service providers related to the proposed relaxation method. It identifies nearby service providers from databases and online resources based on the user's location and other information. For example, it searches for information on "nearby aromatherapy salons" or "yoga classes."

[1178] 5. Presentation of results

[1179] The terminal visually presents information about the service provider to the user. For example, a text such as "Click here for the nearest aromatherapy salon" and a link are displayed on the user's screen.

[1180] 6. Receiving and Analyzing Feedback

[1181] The user tries out the provided relaxation methods and inputs their impressions and effects into the device. For example, they might input, "I tried aromatherapy and it helped me relax a little." This feedback is sent from the device to the server, which then analyzes it.

[1182] 7. Improving proposal accuracy

[1183] The server analyzes the received feedback and applies the results to the machine learning model, which improves the accuracy of relaxation method suggestions from the next time onwards.

[1184] Prompt Sentence Examples

[1185] "When a user types, 'I've been busy at work lately and feeling stressed,' create a system that suggests appropriate relaxation methods and nearby service providers. Specifically, implement a way to extract keywords using natural language processing, search for information on the suggested relaxation methods and related service providers, and visually present them to the user."

[1186] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1187] Step 1: Receiving User Input

[1188] The user inputs the stress, anxiety, and specific worries they are experiencing into the device in text format, and the device receives this input as text data.

[1189] input:

[1190] Text data entered by the user, such as "I've been busy at work lately and feeling stressed."

[1191] output:

[1192] Text data sent to the device.

[1193] Specific behavior:

[1194] The user enters "I've been busy at work lately and it's stressful" into the input field of the terminal and presses the send button.

[1195] The terminal transmits this text data to the server.

[1196] Step 2: Analyzing the text data

[1197] The server analyzes the received text data, tokenizing the input and extracting keywords using natural language processing techniques, specifically using the spaCy and NLTK libraries.

[1198] input:

[1199] Text data sent from the device: "Work has been busy lately and I'm feeling stressed."

[1200] output:

[1201] Tokenized keywords (e.g., "work" and "stress").

[1202] Specific behavior:

[1203] The server breaks down the text data into tokens and extracts important keywords.

[1204] For example, it breaks down the data into tokens such as "recently," "work," "busy," "stressed," and "piling up," and then extracts "work" and "stress" from these as keywords.

[1205] Step 3: Suggested relaxation techniques

[1206] The server then lists appropriate relaxation methods based on the extracted keywords and analysis results, and suggests these methods to the user.

[1207] input:

[1208] Extracted keywords (e.g., "work," "stress").

[1209] output:

[1210] A list of suggested relaxation techniques (e.g., "deep breathing," "aromatherapy," "yoga").

[1211] Specific behavior:

[1212] The server searches a database for relaxation methods related to the keywords and lists them in order of priority.

[1213] For example, select methods such as "deep breathing," "aromatherapy," and "yoga" and create a list.

[1214] Step 4: Find a service provider

[1215] The server searches for service providers related to the proposed relaxation method using databases and online resources, taking into account the user's location.

[1216] input:

[1217] A list of suggested relaxation techniques (e.g., "deep breathing," "aromatherapy," "yoga"), and the user's location.

[1218] output:

[1219] Information about service providers (e.g., "aromatherapy salons near me" or "yoga classes").

[1220] Specific behavior:

[1221] The server searches for nearby service providers from databases and online resources based on the user's location information.

[1222] Collect information on salons related to "aromatherapy" and classes related to "yoga."

[1223] Step 5: Presenting the results

[1224] The terminal visually presents the information of the service provider received from the server to the user.

[1225] input:

[1226] Information about the service provider sent from the server (e.g., "Here is the nearest aromatherapy salon").

[1227] output:

[1228] Service provider information displayed on the user's device.

[1229] Specific behavior:

[1230] The device displays text such as "Click here for nearby aromatherapy salons" and a link on the user's screen.

[1231] Detailed information on relaxation methods and how to access them are also displayed.

[1232] Step 6: Receive and analyze feedback

[1233] The user performs the provided relaxation method and inputs their impressions and effects into the device, which then sends the feedback to the server, which then analyzes it.

[1234] input:

[1235] User-supplied feedback (e.g., "I tried aromatherapy and it helped me relax a bit").

[1236] output:

[1237] Parsed feedback data.

[1238] Specific behavior:

[1239] The user types "I tried aromatherapy and it helped me relax a bit" into the device and presses the send button.

[1240] The terminal sends this feedback to the server.

[1241] Step 7: Improving the accuracy of the proposals

[1242] The server analyzes the received feedback and applies the results to the machine learning model, which improves the accuracy of relaxation method suggestions from the next time onwards.

[1243] input:

[1244] The feedback data received.

[1245] output:

[1246] Improved machine learning models for more accurate recommendations.

[1247] Specific behavior:

[1248] The server analyzes the feedback data and evaluates the effectiveness of the relaxation methods.

[1249] Based on the evaluation results, the machine learning model will be retrained to improve the accuracy of proposals from next time onwards.

[1250] (Application example 1)

[1251] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1252] Users of autonomous vehicles want to maintain their mental health by having opportunities to relax while traveling or waiting. However, it is difficult to suggest appropriate relaxation methods in real time according to the user's mental state and provide high-quality relaxation services based on those suggestions. It is also necessary to utilize the user's location information to accurately identify and guide nearby service providers. A system that solves these problems and makes the autonomous vehicle experience more comfortable and healthy is needed.

[1253] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1254] In this invention, the server includes means for receiving input from a user, means for analyzing the received input and identifying the user's mental state, means for suggesting an appropriate relaxation method based on the analysis result, means for searching for providers related to the suggested relaxation method, means for presenting information about the providers to the user, means for receiving and analyzing feedback from the user, means for improving the accuracy of the suggested relaxation method based on the feedback, means for providing a video guide or audio guide corresponding to the identified relaxation method, and means for using smart glasses in the autonomous vehicle to support the driver's mental health based on the specified relaxation method, thereby enabling the user to enjoy a relaxing environment in the car while receiving mental health care in real time.

[1255] Word definition

[1256] The "means for receiving input from the user" is an interface for receiving text data or voice data input by the user through the terminal.

[1257] "Means for analyzing received input content and identifying the user's mental state" refers to a function that analyzes the received user input data using natural language processing technology, etc., to identify the type of stress or anxiety the user is experiencing.

[1258] The "means for suggesting appropriate relaxation methods" is a function that selects and suggests relaxation activities suitable for the user based on the analyzed mental state.

[1259] The "means for searching for providers related to the proposed relaxation method" is a function for searching and identifying businesses or facilities that provide the proposed relaxation activity from a database.

[1260] The "means for presenting provider information to the user" is an interface that displays detailed information of the searched provider to the user visually or audibly.

[1261] The "means for receiving and analyzing feedback from users" is a function for receiving and analyzing input data such as the user's impressions and effects after trying a relaxation method.

[1262] "Means for improving the accuracy of relaxation method suggestions based on feedback" refers to a function that reflects received feedback data in the learning algorithm to improve the accuracy of suggestions from the next time onwards.

[1263] The "means for providing video guidance or audio guidance corresponding to the identified relaxation method" is a function that provides the user with video guidance or audio guidance for carrying out the proposed relaxation method.

[1264] "Means for using smart glasses in autonomous vehicles" refers to a function that uses smart glasses installed in autonomous vehicles to display and guide information on relaxation methods through the user's gaze or voice input.

[1265] MODE FOR CARRYING OUT THE INVENTION

[1266] This invention relates to a relaxation concierge system that uses smart glasses installed in autonomous vehicles to support the user's mental health. This system receives and analyzes input from the user to suggest appropriate relaxation methods, and further identifies and presents service providers based on those methods. It also has an algorithm that receives feedback after performing a relaxation method and improves its accuracy.

[1267] Hardware Configuration

[1268] Terminal: Smart glasses installed in autonomous vehicles. A device that collects and displays data through the user's gaze and voice input.

[1269] Server: A central server that analyzes data and proposes relaxation methods. Operated as a remote cloud server.

[1270] Network: A communications network for connecting terminals and servers. It uses the Internet or a dedicated communications network.

[1271] Software Configuration

[1272] Natural Language Processing Library (NLPProcessor): Analyzes user input, extracts keywords, and identifies mental states.

[1273] Location Service API (LocationServiceAPI): Obtains the user's location information and searches for nearby relaxation providers.

[1274] Relaxation Provider API (RelaxationServiceAPI): Provides relaxation method suggestions and related provider information to the server.

[1275] Processing flow

[1276] The user inputs text or voice through the smart glasses, saying, "Recently, work has been busy and I'm feeling stressed." This input is received by the device and sent to the server via the network. The server uses an NLP processor to analyze the input and extract keywords such as "work" and "stress." This identifies the user's mental state as "caused by work-related stress."

[1277] Next, the server executes a relaxation method suggestion function, suggesting techniques such as "deep breathing," "aromatherapy," and "yoga" to the user. A video guide or audio guide corresponding to the suggested relaxation method is displayed on the smart glasses. In addition, information on nearby relaxation providers is obtained via a location information service API. For example, "aromatherapy salons" and "yoga studios" near the user's current location are displayed.

[1278] The user performs the relaxation technique and then inputs feedback through the smart glasses. This feedback is then sent back to the server for analysis. The server then uses the collected feedback to train the suggestion algorithm and improve the accuracy of future suggestions.

[1279] Specific examples

[1280] User input: "Work has been busy and stressful lately."

[1281] Suggested relaxation methods: "Deep breathing," "Aromatherapy," and "Yoga"

[1282] Suggested prompt examples:

[1283] Please suggest ways to reduce the stress you feel at work.

[1284] "Tell me how to relax"

[1285] "Find nearby relaxation facilities"

[1286] This system provides users with relaxation techniques while driving or waiting, enabling them to maintain their mental health. Feedback is also used based on the results of practicing the provided relaxation techniques, improving the accuracy of the suggestions.

[1287] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1288] Detailed explanation of the processing steps

[1289] Step 1:

[1290] Input: A user types or speaks through smart glasses, "Work has been busy and stressful lately."

[1291] Action: The device receives this input and sends it over the network to the server.

[1292] Data processing and calculation: Receives data (text or voice data) input from the smart glasses and sends it to the server as digital data.

[1293] Output: The input data is sent to the server.

[1294] Step 2:

[1295] Input: User input data sent from the device.

[1296] How it works: The server uses the NLPProcessor to parse the input data it receives and extract important keywords.

[1297] Data processing and calculation: Using natural language processing, the input data is tokenized and keywords such as "work" and "stress" are extracted.

[1298] Output: Extracted keywords.

[1299] Step 3:

[1300] Input: Extracted keywords (e.g., "work" and "stress").

[1301] Operation: The server runs an algorithm to suggest relaxation methods based on the extracted keywords.

[1302] Data processing and calculation: Keywords are compared with past databases to select the most suitable relaxation method (e.g., "deep breathing," "aromatherapy," "yoga").

[1303] Output: A list of suggested relaxation methods.

[1304] Step 4:

[1305] Input: A list of suggested relaxation methods.

[1306] How it works: The server prepares video and audio guides corresponding to relaxation methods and sends them to the smart glasses.

[1307] Data processing and calculation: Searching and selecting video and audio files corresponding to the relaxation methods.

[1308] Output: Video guide and audio guide data.

[1309] Step 5:

[1310] Input: A list of suggested relaxation methods and the user's location.

[1311] How it works: The server uses the LocationService API to search for nearby relaxation providers and sends them to the smart glasses.

[1312] Data processing and calculation: Based on the user's location information, nearby facilities that offer the proposed relaxation methods are identified and information is collected.

[1313] Output: A list of nearby relaxation providers.

[1314] Step 6:

[1315] Input: Video and audio guide data, and a list of nearby relaxation providers.

[1316] How it works: The device displays this information on the smart glasses display.

[1317] Data processing and calculation: Present data to the user visually or audibly.

[1318] Output: The user receives information through the smart glasses that encourages them to practice relaxation techniques and provides guidance on nearby facilities.

[1319] Step 7:

[1320] Input: After the user performs the relaxation technique, they enter feedback about their experience.

[1321] Operation: The terminal receives the user's feedback and transmits it to the server via the network.

[1322] Data processing and calculation: Feedback data is collected and stored in a database.

[1323] Output: User feedback is sent to the server.

[1324] Step 8:

[1325] Input: User feedback.

[1326] How it works: The server analyzes the feedback and uses it to improve the accuracy of the proposed algorithm.

[1327] Data processing and calculation: Improve the accuracy of suggestions by re-training the algorithm based on feedback data.

[1328] Output: Relaxation suggestions will be performed more accurately from the next time onwards.

[1329] This series of processes allows users to receive efficient mental care while in an autonomous vehicle.

[1330] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1331] This invention provides a system that combines an emotion engine to support users' mental health care. This system analyzes input from the user and suggests appropriate relaxation methods based on the user's emotional state. It also has the function of searching for service providers related to the suggested relaxation methods and presenting them to the user.

[1332] First, the user inputs their daily stress, anxiety, and specific worries into the device. The system then incorporates an emotion engine to recognize emotions from the user's facial expressions and voice. The device then transmits the user's text input, facial expression analysis, and voice analysis data to the server.

[1333] The server analyzes the received text data using natural language processing technology to extract key keywords. It also analyzes the user's emotions recognized by the emotion engine and compares them with the mental state extracted from the text data.

[1334] For example, consider the case where a user inputs "I've been busy at work lately and feeling stressed," and emotions of tension and anxiety are recognized from their facial expressions and voice. The server extracts the keywords "work" and "stress," and by combining these with the emotions identified by the emotion engine as "tension" and "anxiety," identifies the user's mental state as "stress, tension, and anxiety due to work."

[1335] Next, the server will list appropriate relaxation methods based on the identified mental state, such as "deep breathing," "aromatherapy," and "yoga." These relaxation methods are intended to reduce the tension and anxiety identified by the emotion engine.

[1336] The server then searches for service providers related to the selected relaxation method, identifying nearby aromatherapy salons and yoga classes based on the user's location information from affiliated databases and online resources, and transmitting information about the identified service providers to the device.

[1337] The terminal visually presents the information about the service provider to the user. For example, the terminal displays a text message such as "Click here for the nearest aromatherapy salon" and a link on the user's screen.

[1338] After a user visits a salon or class and tries out a relaxation method, the device receives feedback about the effects and impressions of the experience. For example, if a user types, "I tried aromatherapy and it helped me relax a little," the device sends this feedback to the server.

[1339] Finally, the server analyzes the received feedback and reflects the analysis results in the emotion engine, which improves the accuracy of relaxation method suggestions from the next time onwards and makes suggestions that are more suited to the user's individual emotional state.

[1340] As a concrete example of this system, if a user inputs "I can't sleep at night and I'm frustrated lately," and the server recognizes fatigue from the user's facial expression and voice, it will combine keywords such as "can't sleep" and "frustrated" with "fatigue" to identify the symptom as "fatigue and frustration due to lack of sleep." In this case, the server will suggest methods such as "aromatherapy" and "relaxation music," and search for and display nearby related service providers.

[1341] In this way, by combining an emotion engine, the present invention provides a system that proposes more accurate relaxation methods based on the user's emotional state and quickly searches for and presents service providers, allowing users to easily find the relaxation method that best suits them and effectively relieve stress.

[1342] The processing flow will be explained below.

[1343] Step 1:

[1344] The user inputs information about their stress and anxiety, for example, "Work has been busy lately and I'm feeling stressed."

[1345] Step 2:

[1346] The device receives the user's input as text data, and also acquires the user's facial expression and voice data.

[1347] Step 3:

[1348] The terminal transmits the received text data, facial expression data, and voice data to the server.

[1349] Step 4:

[1350] The server receives the data sent from the terminal.

[1351] Step 5:

[1352] The server uses natural language processing technology to analyze the text data and extract key keywords. In this case, the keywords extracted are "work" and "stress."

[1353] Step 6:

[1354] The server uses an emotion engine to analyze the received facial expression and voice data and recognize the user's emotions, such as "tension" or "anxiety."

[1355] Step 7:

[1356] The server compares the keywords extracted from the text data with the emotions recognized by the emotion engine to identify the user's mental state. Specifically, it identifies "stress, tension, and anxiety due to work."

[1357] Step 8:

[1358] Based on the identified mental state, the server will then list appropriate relaxation methods, such as "deep breathing," "aromatherapy," and "yoga."

[1359] Step 9:

[1360] The server searches for service providers related to the selected relaxation method, searching a database for nearby aromatherapy salons and yoga classes based on the user's location information.

[1361] Step 10:

[1362] The server prepares information about the service providers obtained as a result of the search and transmits it to the terminal.

[1363] Step 11:

[1364] The terminal receives the information about the service provider sent from the server.

[1365] Step 12:

[1366] The terminal visually displays the received information about the service provider to the user, providing a text such as "Click here for nearby aromatherapy salons" and a link.

[1367] Step 13:

[1368] The user can then take action based on the displayed information about relaxation methods and service providers, for example, visiting a nearby aromatherapy salon.

[1369] Step 14:

[1370] After performing the relaxation method, the user inputs the results as feedback into the terminal, for example, "I tried aromatherapy and it helped me relax a little."

[1371] Step 15:

[1372] The terminal receives feedback from the user and transmits it to the server.

[1373] Step 16:

[1374] The server receives the feedback sent from the terminal.

[1375] Step 17:

[1376] The server analyzes the feedback content and evaluates the effectiveness of the relaxation method.

[1377] Step 18:

[1378] The server reflects the analysis results in the emotion engine, improving the accuracy of relaxation method suggestions from the next time onwards.

[1379] This series of steps allows users to easily find the relaxation method that best suits them and provides effective stress relief measures based on their emotions and mental state.

[1380] Example 2

[1381] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1382] In modern society, users experience a great deal of stress and anxiety due to their daily lives and work. To properly care for these mental health issues, a system is needed that can accurately assess the user's condition and suggest optimal relaxation methods. It is also necessary to analyze the user's emotional state in real time and respond quickly according to that state. Furthermore, it is necessary to immediately provide information on service providers related to the suggested relaxation methods, thereby reducing the burden on the user. Conventional systems have had difficulty meeting these requirements, so providing customized mental health care to users has become a challenge.

[1383] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1384] In this invention, the server includes means for receiving input from a user, means for analyzing the received input and identifying the user's mental state, means for suggesting an appropriate relaxation method based on the analysis results, means for searching for service providers related to the suggested relaxation method, means for presenting information about the service providers to the user, means for collecting the user's facial expression data and voice data and analyzing them using an emotion engine, means for receiving and analyzing feedback from the user, and means for improving the accuracy of suggested relaxation methods based on the feedback. This enables accurate suggestions of relaxation methods based on the user's individual mental state and rapid search and presentation of related service providers.

[1385] "User input" is information provided by a user to a terminal through text, voice, or facial expressions.

[1386] "Mental state" refers to the mental and psychological state of the user, such as stress, anxiety, and worries.

[1387] "Relaxation methods" are methods such as deep breathing, aromatherapy, and yoga that are proposed as means for improving the user's mental state.

[1388] "Service provider" refers to a person or facility that provides services related to relaxation methods.

[1389] "Suggestion accuracy" indicates the degree of accuracy and relevance when suggesting relaxation methods appropriate for the user's mental state.

[1390] An "emotion engine" is a technology or algorithm that analyzes a user's facial expressions and voice data to identify their emotional state.

[1391] "Natural language processing technology" is a technology for analyzing user input text and understanding its meaning.

[1392] "Feedback" refers to information about impressions and effects provided by users after using the proposed relaxation method or related service.

[1393] "Location information" is information that indicates the geographical location where the user currently resides.

[1394] This invention is a system that supports users' mental health care and uses technology that combines an emotion engine. Users input their daily stress, anxiety, and specific worries into a terminal, and facial expression and voice data are also collected. This input data is sent to a server and analyzed.

[1395] Hardware and software used

[1396] This system uses the following hardware and software.

[1397] Terminal: The device through which the user provides input (e.g., smartphone, PC).

[1398] Emotion engine: Technology for analyzing the user's facial expressions and voice (OpenCV, IBM Watson, Google Cloud Speech-to-Text, etc.).

[1399] Server: A computer system for analyzing and processing data.

[1400] Natural language processing technologies: Technologies for analyzing user text input (NLTK library, generative AI models such as GPT-3).

[1401] System operation explanation

[1402] 1. User Input

[1403] Users use the device to input their daily stress, anxiety, and worries in text format, while facial expression and voice data are also collected at the same time.

[1404] Example: A user types, "I've been busy at work lately and feeling stressed," and their facial expressions and voice are also recorded.

[1405] 2. Sending data to the server

[1406] The terminal transmits the input text data, facial expression data, and voice data to the server.

[1407] Technical example: A smartphone or PC sends data to a server as an HTTP request.

[1408] 3. Text Data Analysis

[1409] The server uses natural language processing technology to analyze the received text data and extract key keywords.

[1410] Technologies used: Python's NLTK library and generative AI models (such as GPT-3).

[1411] Example: Extract the keywords "work" and "stress" from the analysis results of the text "I've been busy at work lately and feeling stressed."

[1412] 4. Emotion Data Analysis

[1413] The server uses an emotion engine to analyze facial and voice data to identify the user's emotional state (e.g., tension, anxiety).

[1414] Technologies used: OpenCV for facial expression recognition, IBM Watson and Google Cloud Speech-to-Text for voice emotion recognition.

[1415] For example, "tension" can be recognized from facial expressions, and "anxiety" can be recognized from voice.

[1416] 5. Relaxation suggestions

[1417] Based on the analysis results, the server identifies the user's mental state and suggests appropriate relaxation methods (deep breathing, aromatherapy, yoga, etc.).

[1418] Example: Suggest "deep breathing," "aromatherapy," and "yoga" to combat "stress, tension, and anxiety caused by work."

[1419] 6. Search for a service provider

[1420] The server searches affiliated databases and online resources for service providers related to relaxation techniques.

[1421] The search for service providers is carried out based on the user's location information.

[1422] For example: Identify nearby aromatherapy salons or yoga classes.

[1423] 7. Presentation of Service Provider Information

[1424] The terminal visually presents the search results to the user and provides information on nearby service providers.

[1425] Example: Displaying text and a link such as "Click here for aromatherapy salons near you" on the user screen.

[1426] 8. User Feedback

[1427] After actually using the service, the user inputs the results and impressions into the terminal as feedback.

[1428] Example: User types, "I tried aromatherapy and it helped me relax a bit."

[1429] 9. Incorporating Feedback

[1430] The server analyzes the received feedback and reflects it in the emotion engine, thereby improving the accuracy of relaxation method suggestions from the next time onwards.

[1431] Technical example: Integrating feedback into the system by retraining machine learning models.

[1432] Through the above series of steps, the present invention provides a system that effectively supports users' mental health care. By combining specific technologies and operations, it is possible to accurately suggest relaxation methods based on the user's emotional state and quickly search for and present service providers.

[1433] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1434] Step 1:

[1435] User Input and Data Capture

[1436] Users use a device to input their daily stress, anxiety, and worries in text format. The system also collects facial and voice data. The input data includes text, facial images, and voice files.

[1437] Input: Text data, facial expression data, voice data

[1438] How it works: The user types, "I've been busy at work lately and feeling stressed," and their facial expressions and voice are also recorded. The device's camera and microphone capture the user's facial expressions and voice in real time.

[1439] Output: Input data (text, facial expressions, voice)

[1440] Step 2:

[1441] Sending data to the server

[1442] The terminal transmits the collected text data, facial expression data, and voice data to the server.

[1443] Input: Input data (text, facial expressions, voice)

[1444] How it works: Your smartphone or PC sends data to the server as an HTTP request, which is then formatted and encoded.

[1445] Output: Received data on the server (text, facial expressions, voice)

[1446] Step 3:

[1447] Text data analysis

[1448] The server analyzes the received text data using natural language processing technology and extracts key keywords.

[1449] Input: Received text data on the server

[1450] How it works: It uses Python's NLTK library and generative AI models (such as GPT-3) to extract important keywords from text.

[1451] Output: Extracted keywords (e.g., "work" and "stress")

[1452] Step 4:

[1453] Emotional Data Analysis

[1454] The server uses an emotion engine to analyze facial expression and voice data to identify the user's emotional state.

[1455] Input: Received facial expression data and voice data on the server

[1456] Operation: Classifies emotional states using OpenCV for facial expression recognition and IBM Watson or Google Cloud Speech-to-Text for voice emotion recognition. Example: "tension" or "anxiety."

[1457] Output: Identified emotion (e.g., "tension" or "anxiety")

[1458] Step 5:

[1459] Suggestions for relaxation methods

[1460] The server combines the analyzed keywords with the user's emotional state to identify the user's mental state and list appropriate relaxation methods.

[1461] Input: extracted keywords, identified sentiments

[1462] How it works: It detects the user's mental state and runs an algorithm that selects relaxation methods (e.g., "deep breathing," "aromatherapy," "yoga") based on that.

[1463] Output: A list of suggested relaxation techniques (e.g., "deep breathing," "aromatherapy," "yoga")

[1464] Step 6:

[1465] Search for a service provider

[1466] The server searches affiliated databases and online resources for service providers related to relaxation methods, and identifies the service provider based on the user's location information.

[1467] Input: List of relaxation methods, user location

[1468] How it works: Searches for nearby service providers (e.g., aromatherapy salons or yoga classes) through affiliated databases and APIs, and obtains rankings and reviews.

[1469] Output: List of relevant service providers with details

[1470] Step 7:

[1471] Present service provider information

[1472] The terminal visually presents information about service providers to the user based on the search results.

[1473] Input: List of relevant service providers and details

[1474] Operation: Displays information about service providers on the user interface. For example, it displays text such as "Click here for nearby aromatherapy salons," links, and map information on the screen.

[1475] Output: Service provider information presented to the user

[1476] Step 8:

[1477] User Feedback

[1478] After actually using the service, the user inputs feedback about its effectiveness and impressions into the terminal.

[1479] Input: User feedback data (text format)

[1480] What it does: Displays a feedback form and allows the user to fill it in. For example, the user might type, "I tried aromatherapy and it helped me relax a bit."

[1481] Output: Feedback data

[1482] Step 9:

[1483] Reflecting feedback

[1484] The server analyzes the received feedback and reflects it in the emotion engine, thereby improving the accuracy of relaxation method suggestions from the next time onwards.

[1485] Input: User feedback data

[1486] How it works: The feedback data is analyzed and used as data to retrain the emotion engine model, thereby improving the accuracy of the proposed algorithm.

[1487] Output: Emotion engine and proposed algorithm with improved accuracy

[1488] (Application example 2)

[1489] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1490] Conventional mental healthcare systems estimate a user's mental state based on text input and suggest relaxation methods. However, in order to recognize the user's emotional state with greater accuracy, it is necessary to incorporate facial expression and voice analysis. Furthermore, systems lack the ability to not only suggest appropriate relaxation methods but also to quickly search for and present relevant service providers. Furthermore, systems lack the ability to identify service providers based on the user's location information or to incorporate user feedback to improve the accuracy of future suggestions.

[1491] The identification process performed by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving input from the user, means for analyzing the received input and identifying the user's mental state, means for recognizing the user's emotional state through facial expression analysis and voice analysis, means for suggesting an appropriate relaxation method based on the analysis results, means for searching for service providers related to the suggested relaxation method, means for presenting information about the service providers to the user, means for receiving and analyzing feedback from the user, and means for improving the accuracy of relaxation method suggestions based on the feedback. This makes it possible to recognize the user's emotional state with higher accuracy and quickly identify and present appropriate relaxation methods and related service providers. Furthermore, using the feedback improves the accuracy of suggestions from the next time onwards.

[1492] "Means for receiving input from a user" refers to a device that allows a user to input text or voice and software for processing the input.

[1493] The "means for analyzing the received input content and identifying the user's mental state" refers to an algorithm and processing device that analyzes the text and voice data entered by the user and infers the user's emotions and psychological state from the content.

[1494] "Means for recognizing a user's emotional state through facial expression analysis and voice analysis" refers to technology that analyzes a user's facial expressions and tone of voice and recognizes their emotional state from the results.

[1495] The "means for suggesting an appropriate relaxation method based on the analysis results" refers to an algorithm and processing device for selecting and suggesting a relaxation method based on the analysis results of the user's mental and emotional state.

[1496] "Means for searching for service providers related to the proposed relaxation method" refers to a database and search engine for searching and identifying businesses and facilities that provide services according to the proposed relaxation method.

[1497] The "means for presenting information of service providers to a user" refers to a user interface and software for displaying the searched information of service providers on a user's device.

[1498] The "means for receiving and analyzing feedback from the user" refers to a means for collecting and analyzing the user's impressions and experiences after implementing the relaxation method.

[1499] The "means for improving the accuracy of relaxation method suggestions based on feedback" refers to an algorithm and processing device that uses the results of analyzing user feedback to improve the accuracy of relaxation method suggestions from the next time onwards.

[1500] The "means for identifying nearby service providers based on the user's location information" refers to a geographic information system and search engine for identifying nearby relevant service providers based on the user's current location information.

[1501] "Means for analyzing user input using natural language processing technology" refers to natural language processing algorithms and software that analyze text data entered by users and extract key keywords and emotions.

[1502] The present invention is a system for supporting a user's mental health care. This system receives input data from the user, analyzes it, and suggests appropriate relaxation methods. It also has the function of searching for service providers related to the suggested relaxation methods and presenting them to the user. The system also receives feedback and improves the accuracy of relaxation method suggestions from the next time onwards.

[1503] Hardware and Software Examples

[1504] 1. Hardware:

[1505] Smartphones (iOS and Android devices)

[1506] server

[1507] 2. Software:

[1508] Natural Language Processing: Google Cloud Natural Language API

[1509] Facial expression analysis: Amazon Rekognition

[1510] Speech analysis: Google Cloud Speech-to-Text API

[1511] Location search: Google Maps API

[1512] Data processing and calculation flow

[1513] 1. Receiving user input:

[1514] Users use a smartphone application to input their stress and anxiety levels via text or voice, while facial expression data is also collected via the camera.

[1515] 2. Analysis of input data:

[1516] The text data is analyzed using the Google Cloud Natural Language API to extract key keywords and the user's mental state.

[1517] Facial expression data is analyzed using Amazon Rekognition to identify the user's emotions.

[1518] The audio data is converted to text using the Google Cloud Speech-to-Text API, and then sentiment analysis is performed using the Google Cloud Natural Language API.

[1519] 3. Relaxation suggestions:

[1520] Based on the analysis results, a database is consulted to suggest appropriate relaxation techniques (e.g., deep breathing, aromatherapy, yoga).

[1521] 4. Search and present service providers:

[1522] Service providers related to the proposed relaxation method (e.g., aromatherapy salons, yoga classes) are searched for using the user's location information using the Google Maps API and presented to the user.

[1523] 5. Feedback Collection and Analysis:

[1524] After trying out a relaxation technique, users can enter their impressions and feedback through the application. This feedback is analyzed by the server and used to improve the accuracy of future suggestions.

[1525] Specific examples

[1526] 1. User Input:

[1527] The user types, "Work has been busy and stressful lately."

[1528] Facial expression analysis recognizes the emotion of "tension" from the user's face.

[1529] Voice analysis identifies the emotion of "anxiety" from voice.

[1530] 2. Relaxation suggestions:

[1531] The server suggests "deep breathing," "aromatherapy," and "yoga" as suitable relaxation methods.

[1532] 3. Service Provider Presentation:

[1533] The text and link "Here are the nearest aromatherapy salons" will appear on the app screen.

[1534] Prompt Sentence Examples

[1535] User input: "Work has been busy and stressful lately."

[1536] Recognized emotional state: Facial expression analysis "tension", voice analysis "anxiety"

[1537] Extracted keywords: "work", "stress"

[1538] Suggested relaxation methods: "Deep breathing", "Aromatherapy", "Yoga"

[1539] Search for nearby service providers: Google Maps API

[1540] This allows for more accurate recognition of the user's emotional state, enabling the system to suggest appropriate relaxation methods and quickly search for and present relevant service providers. Furthermore, by using user feedback, the system can improve the accuracy of suggestions from the next time onwards, providing a more personalized experience.

[1541] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1542] Step 1:

[1543] A user inputs text or voice using a smartphone application.

[1544] Input: User text input, voice input, and facial expression data

[1545] Specific behavior:

[1546] The user inputs the stress or anxiety they are experiencing through text or voice.

[1547] Facial expression data is collected via a smartphone camera.

[1548] Output: Input data (text, voice, facial expression)

[1549] Step 2:

[1550] The terminal transmits the collected input data to the server.

[1551] Input: Input data (text, voice, facial expression)

[1552] Specific behavior:

[1553] The terminal transmits the text data as is, the voice data as a voice file, and the facial expression data as an image to the server.

[1554] Output: Input data sent to the server

[1555] Step 3:

[1556] The server analyzes the received text data using the Google Cloud Natural Language API to extract key keywords and mental states.

[1557] Input: Text data

[1558] Specific behavior:

[1559] The server calls the Google Cloud Natural Language API to parse the text data.

[1560] Extract key keywords (e.g., "work," "stress") and the user's mental state.

[1561] Output: Key keywords and mental states

[1562] Step 4:

[1563] The server analyzes the facial expression data using Amazon Rekognition and recognizes the user's emotions.

[1564] Input: facial expression data (image)

[1565] Specific behavior:

[1566] The server calls Amazon Rekognition to analyze the facial expression data.

[1567] Recognize the user's emotional state (e.g., "tension") from facial expressions.

[1568] Output: Perceived emotional state

[1569] Step 5:

[1570] The server converts the voice data into text using the Google Cloud Speech-to-Text API, and then analyzes the sentiment using the Google Cloud Natural Language API.

[1571] Input: Audio data

[1572] Specific behavior:

[1573] The server calls the Google Cloud Speech-to-Text API to convert the audio data into text.

[1574] The converted text data is then analyzed again using the Google Cloud Natural Language API to identify sentiment.

[1575] Output: Perceived emotional state

[1576] Step 6:

[1577] The server refers to a database to suggest appropriate relaxation methods based on the analysis results.

[1578] Input: Primary keywords, perceived emotional state

[1579] Specific behavior:

[1580] Based on the analysis results, the most suitable relaxation method for the user (e.g., "deep breathing," "aromatherapy," "yoga") is selected from a database.

[1581] Output: Proposed relaxation method

[1582] Step 7:

[1583] The server uses the Google Maps API to search for nearby service providers related to the proposed relaxation method.

[1584] Input: Proposed relaxation method, user location

[1585] Specific behavior:

[1586] Call the Google Maps API to search for service providers based on the user's location information.

[1587] Identify service providers suitable for the proposed relaxation method.

[1588] Output: Service provider information

[1589] Step 8:

[1590] The terminal presents information about the service provider to the user.

[1591] Input: Service provider information

[1592] Specific behavior:

[1593] Display information about the service provider (e.g., "Click here for the nearest aromatherapy salon") in text and link format on the user's app screen.

[1594] Output: Information presented to the user

[1595] Step 9:

[1596] After the user tries the relaxation techniques, they enter feedback through the application.

[1597] Input: User feedback (text)

[1598] Specific behavior:

[1599] The user inputs the effects and impressions of the relaxation method in text format.

[1600] Output: Input feedback

[1601] Step 10:

[1602] The terminal transmits the feedback from the user to the server.

[1603] Input: User feedback

[1604] Specific behavior:

[1605] The terminal transmits the feedback text data to the server.

[1606] Output: Feedback sent to the server

[1607] Step 11:

[1608] The server analyzes the received feedback and updates the database to improve the accuracy of relaxation method suggestions from the next time onwards.

[1609] Input: Feedback data

[1610] Specific behavior:

[1611] The feedback data will be analyzed and the results will be used to update the database to improve the accuracy of relaxation method suggestions.

[1612] Output: Updated database

[1613] The above processing steps enable highly accurate recognition of the user's emotional state, suggesting appropriate relaxation methods, and quickly searching and presenting relevant service providers. Furthermore, by using feedback, the accuracy of suggestions can be improved in future visits, providing a more personalized experience.

[1614] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1615] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1616] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1617] [Fourth embodiment]

[1618] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1619] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1620] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1621] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1622] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1623] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1624] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1625] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1626] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1627] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1628] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1629] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1630] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1631] This invention provides a system that proposes appropriate relaxation methods and stress-relief activities for users with mental health concerns. This system analyzes user input, proposes relaxation methods based on the results, and searches for and presents related service providers.

[1632] First, the user inputs into the device the stress, anxiety, and specific worries they experience in their daily lives. The device then sends this input as text data to the server. The server then analyzes the received text data using natural language processing technology. Specifically, it tokenizes the input and extracts keywords.

[1633] For example, consider the case where a user enters, "Recently, I've been busy at work and feeling stressed." In this case, the server extracts keywords such as "work" and "stress" and identifies the user's mental state as "caused by stress from work."

[1634] Next, the server executes a means to suggest relaxation methods based on the identified mental state. For example, it lists several relaxation methods as candidates, such as "deep breathing," "aromatherapy," and "yoga." This list prioritizes those most relevant to the user's input.

[1635] The server then executes a search for service providers related to the proposed relaxation method, identifying nearby service providers based on the user's location information from affiliated databases and online resources, thereby providing the user with information on nearby aromatherapy salons, yoga classes, relaxation facilities, etc.

[1636] The terminal visually presents the information about the service provider to the user. For example, the terminal displays a text message such as "Click here for the nearest aromatherapy salon" and a link on the user's screen.

[1637] After a user visits a salon or class and tries out a relaxation method, the device receives feedback about the effects and impressions of the experience. For example, if a user types, "I tried aromatherapy and it helped me relax a little," the device sends this feedback to the server.

[1638] Finally, the server analyzes the received feedback and applies it to the system's learning algorithm, improving the accuracy of future relaxation suggestions and making them more tailored to the user's needs.

[1639] In this way, the present invention is a system that receives input from the user, analyzes it, and suggests appropriate relaxation methods and affiliated businesses, providing a means to easily relieve stress without being noticed by those around you.

[1640] The processing flow will be explained below.

[1641] Step 1:

[1642] The user inputs information about their stress or anxiety. For example, they might input, "Work has been busy lately and I'm feeling stressed."

[1643] Step 2:

[1644] The terminal receives input from the user as text data.

[1645] Step 3:

[1646] The terminal transmits the received text data to the server.

[1647] Step 4:

[1648] The server receives the text data sent from the terminal.

[1649] Step 5:

[1650] The server uses natural language processing technology to analyze the text data and extract key keywords. In this case, the keywords extracted are "work" and "stress."

[1651] Step 6:

[1652] The server identifies the user's mental state based on the extracted keywords. In this case, "work-related stress" is identified.

[1653] Step 7:

[1654] The server then lists appropriate relaxation methods based on the identified mental state, such as "deep breathing," "aromatherapy," and "yoga."

[1655] Step 8:

[1656] The server searches for service providers related to the selected relaxation method, searching a database for nearby aromatherapy salons and yoga classes based on the user's location information.

[1657] Step 9:

[1658] The server prepares information about the service providers obtained as a result of the search and transmits it to the terminal.

[1659] Step 10:

[1660] The terminal receives the information about the service provider sent from the server.

[1661] Step 11:

[1662] The terminal displays the received information about the service provider to the user, for example, providing a text such as "Click here for nearby aromatherapy salons" and a link.

[1663] Step 12:

[1664] The user then takes action based on the displayed information about relaxation methods and service providers, for example, visiting a nearby aromatherapy salon.

[1665] Step 13:

[1666] After performing the relaxation method, the user inputs the results into the terminal as feedback, for example, "I tried aromatherapy and it helped me relax a little."

[1667] Step 14:

[1668] The terminal receives feedback from the user and transmits it to the server.

[1669] Step 15:

[1670] The server receives the feedback sent from the terminal.

[1671] Step 16:

[1672] The server analyzes the feedback content and evaluates the effectiveness of the relaxation method.

[1673] Step 17:

[1674] The server reflects the analysis results in the system's learning algorithm, improving the accuracy of proposals from next time onwards.

[1675] Through this series of steps, users can easily find the relaxation method that best suits them and effectively relieve stress.

[1676] Example 1

[1677] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1678] In recent years, the number of people suffering from stress and anxiety has been increasing, but there is a lack of specific relaxation methods to appropriately alleviate these stresses and a means to quickly obtain information on support service providers. Furthermore, there is also a lack of mechanisms to provide feedback on the effectiveness of provided relaxation methods and improve the accuracy of suggestions for future sessions. Therefore, there is a need for a system that not only analyzes user input, suggests appropriate relaxation methods, and finds corresponding service providers, but also analyzes feedback to improve services.

[1679] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1680] In this invention, the server includes means for tokenizing the received input content, extracting keywords, and identifying the user's mental state, means for listing and proposing appropriate relaxation methods based on the extracted keywords and analysis results, and means for searching for service providers related to the proposed relaxation methods using databases and online resources. This allows users to instantly obtain relaxation methods and information on service providers that are suitable for them, and the accuracy of the suggestions is further improved through feedback analysis.

[1681] "User" refers to an individual who uses the system to provide input about stress and anxiety and receive information about suggested relaxation techniques and services.

[1682] "Input content" is text data provided by the user to the system, and mainly includes information related to stress and anxiety.

[1683] "Tokenization" is the process of using natural language processing techniques to break down input content into smaller units (tokens) and convert them into a form that is easier to analyze.

[1684] "Keywords" are important words or phrases extracted from text data that serve as clues to identifying the user's mental state and required relaxation methods.

[1685] "Relaxation techniques" refer to specific measures or activities that are used to reduce a user's stress or anxiety. Examples include deep breathing, aromatherapy, and yoga.

[1686] "Service provider" refers to the facility or professional who implements the proposed relaxation method, such as a salon or classroom.

[1687] A "database" is a collection of information in which the system stores information on relaxation methods and service providers.

[1688] "Online Resources" means data and information accessible via the Internet that may be used to locate a Service Provider.

[1689] "Visually presenting" refers to displaying information in a way that users can understand visually, and primarily includes text, links, maps, etc.

[1690] "Feedback" refers to the effects and impressions of the relaxation method the user has tried, and serves as data to improve the accuracy of suggestions from next time onwards.

[1691] A "machine learning model" is an algorithm that allows the system to learn from data and improve the accuracy of relaxation method suggestions from the next time onwards.

[1692] This invention is a system that suggests relaxation methods and stress-relieving activities to alleviate the mental health concerns of users. This system has the function of analyzing the user's input, suggesting relaxation methods based on the results, and searching for and presenting related service providers.

[1693] Hardware Configuration

[1694] The main hardware required to implement the system is as follows:

[1695] User device: A device through which a user inputs information about stress and anxiety, such as a smartphone or computer.

[1696] Server: A central processing unit for analyzing input information and suggesting relaxation methods and service providers.

[1697] Software Configuration

[1698] The main software required to implement the system is as follows:

[1699] Natural language processing libraries: Used to analyze input text data. Specific examples include spaCy and NLTK.

[1700] Machine learning model: Used to analyze feedback and improve the accuracy of relaxation method suggestions. Specifically, we use scikit-learn and TensorFlow.

[1701] Database: Stores information about service providers related to the proposed relaxation methods. Specifically, MySQL or PostgreSQL can be used.

[1702] Specific operation of the system

[1703] 1. User Input

[1704] The user uses the device to input in text form the stress, anxiety, and specific worries they are feeling. For example, they might input, "Recently, work has been busy and I'm feeling stressed."

[1705] 2. Text Data Analysis

[1706] The device receives the input and sends it as text data to the server. The server then uses natural language processing technology to analyze the text data, tokenize it, and extract keywords. For example, keywords such as "work" and "stress" are extracted.

[1707] 3. Suggestions for relaxation methods

[1708] Based on the analysis results, the server uses these keywords to create a list of appropriate relaxation methods and suggests them to the user, such as "deep breathing," "aromatherapy," and "yoga."

[1709] 4. Search for service providers

[1710] The server searches for service providers related to the proposed relaxation method. It identifies nearby service providers from databases and online resources based on the user's location and other information. For example, it searches for information on "nearby aromatherapy salons" or "yoga classes."

[1711] 5. Presentation of results

[1712] The terminal visually presents information about the service provider to the user. For example, a text such as "Click here for the nearest aromatherapy salon" and a link are displayed on the user's screen.

[1713] 6. Receiving and Analyzing Feedback

[1714] The user tries out the provided relaxation methods and inputs their impressions and effects into the device. For example, they might input, "I tried aromatherapy and it helped me relax a little." This feedback is sent from the device to the server, which then analyzes it.

[1715] 7. Improving proposal accuracy

[1716] The server analyzes the received feedback and applies the results to the machine learning model, which improves the accuracy of relaxation method suggestions from the next time onwards.

[1717] Prompt Sentence Examples

[1718] "When a user types, 'I've been busy at work lately and feeling stressed,' create a system that suggests appropriate relaxation methods and nearby service providers. Specifically, implement a way to extract keywords using natural language processing, search for information on the suggested relaxation methods and related service providers, and visually present them to the user."

[1719] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1720] Step 1: Receiving User Input

[1721] The user inputs the stress, anxiety, and specific worries they are experiencing into the device in text format, and the device receives this input as text data.

[1722] input:

[1723] Text data entered by the user, such as "I've been busy at work lately and feeling stressed."

[1724] output:

[1725] Text data sent to the device.

[1726] Specific behavior:

[1727] The user enters "I've been busy at work lately and it's stressful" into the input field of the terminal and presses the send button.

[1728] The terminal transmits this text data to the server.

[1729] Step 2: Analyzing the text data

[1730] The server analyzes the received text data, tokenizing the input and extracting keywords using natural language processing techniques, specifically using the spaCy and NLTK libraries.

[1731] input:

[1732] Text data sent from the device: "Work has been busy lately and I'm feeling stressed."

[1733] output:

[1734] Tokenized keywords (e.g., "work" and "stress").

[1735] Specific behavior:

[1736] The server breaks down the text data into tokens and extracts important keywords.

[1737] For example, it breaks down the data into tokens such as "recently," "work," "busy," "stressed," and "piling up," and then extracts "work" and "stress" from these as keywords.

[1738] Step 3: Suggested relaxation techniques

[1739] The server then lists appropriate relaxation methods based on the extracted keywords and analysis results, and suggests these methods to the user.

[1740] input:

[1741] Extracted keywords (e.g., "work," "stress").

[1742] output:

[1743] A list of suggested relaxation techniques (e.g., "deep breathing," "aromatherapy," "yoga").

[1744] Specific behavior:

[1745] The server searches a database for relaxation methods related to the keywords and lists them in order of priority.

[1746] For example, select methods such as "deep breathing," "aromatherapy," and "yoga" and create a list.

[1747] Step 4: Find a service provider

[1748] The server searches for service providers related to the proposed relaxation method using databases and online resources, taking into account the user's location.

[1749] input:

[1750] A list of suggested relaxation techniques (e.g., "deep breathing," "aromatherapy," "yoga"), and the user's location.

[1751] output:

[1752] Information about service providers (e.g., "aromatherapy salons near me" or "yoga classes").

[1753] Specific behavior:

[1754] The server searches for nearby service providers from databases and online resources based on the user's location information.

[1755] Collect information on salons related to "aromatherapy" and classes related to "yoga."

[1756] Step 5: Presenting the results

[1757] The terminal visually presents the information of the service provider received from the server to the user.

[1758] input:

[1759] Information about the service provider sent from the server (e.g., "Here is the nearest aromatherapy salon").

[1760] output:

[1761] Service provider information displayed on the user's device.

[1762] Specific behavior:

[1763] The device displays text such as "Click here for nearby aromatherapy salons" and a link on the user's screen.

[1764] Detailed information on relaxation methods and how to access them are also displayed.

[1765] Step 6: Receive and analyze feedback

[1766] The user performs the provided relaxation method and inputs their impressions and effects into the device, which then sends the feedback to the server, which then analyzes it.

[1767] input:

[1768] User-supplied feedback (e.g., "I tried aromatherapy and it helped me relax a bit").

[1769] output:

[1770] Parsed feedback data.

[1771] Specific behavior:

[1772] The user types "I tried aromatherapy and it helped me relax a bit" into the device and presses the send button.

[1773] The terminal sends this feedback to the server.

[1774] Step 7: Improving the accuracy of the proposals

[1775] The server analyzes the received feedback and applies the results to the machine learning model, which improves the accuracy of relaxation method suggestions from the next time onwards.

[1776] input:

[1777] The feedback data received.

[1778] output:

[1779] Improved machine learning models for more accurate recommendations.

[1780] Specific behavior:

[1781] The server analyzes the feedback data and evaluates the effectiveness of the relaxation methods.

[1782] Based on the evaluation results, the machine learning model will be retrained to improve the accuracy of proposals from next time onwards.

[1783] (Application example 1)

[1784] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1785] Users of autonomous vehicles want to maintain their mental health by having opportunities to relax while traveling or waiting. However, it is difficult to suggest appropriate relaxation methods in real time according to the user's mental state and provide high-quality relaxation services based on those suggestions. It is also necessary to utilize the user's location information to accurately identify and guide nearby service providers. A system that solves these problems and makes the autonomous vehicle experience more comfortable and healthy is needed.

[1786] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1787] In this invention, the server includes means for receiving input from a user, means for analyzing the received input and identifying the user's mental state, means for suggesting an appropriate relaxation method based on the analysis result, means for searching for providers related to the suggested relaxation method, means for presenting information about the providers to the user, means for receiving and analyzing feedback from the user, means for improving the accuracy of the suggested relaxation method based on the feedback, means for providing a video guide or audio guide corresponding to the identified relaxation method, and means for using smart glasses in the autonomous vehicle to support the driver's mental health based on the specified relaxation method, thereby enabling the user to enjoy a relaxing environment in the car while receiving mental health care in real time.

[1788] Word definition

[1789] The "means for receiving input from the user" is an interface for receiving text data or voice data input by the user through the terminal.

[1790] "Means for analyzing received input content and identifying the user's mental state" refers to a function that analyzes the received user input data using natural language processing technology, etc., to identify the type of stress or anxiety the user is experiencing.

[1791] The "means for suggesting appropriate relaxation methods" is a function that selects and suggests relaxation activities suitable for the user based on the analyzed mental state.

[1792] The "means for searching for providers related to the proposed relaxation method" is a function for searching and identifying businesses or facilities that provide the proposed relaxation activity from a database.

[1793] The "means for presenting provider information to the user" is an interface that displays detailed information of the searched provider to the user visually or audibly.

[1794] The "means for receiving and analyzing feedback from users" is a function for receiving and analyzing input data such as the user's impressions and effects after trying a relaxation method.

[1795] "Means for improving the accuracy of relaxation method suggestions based on feedback" refers to a function that reflects received feedback data in the learning algorithm to improve the accuracy of suggestions from the next time onwards.

[1796] The "means for providing video guidance or audio guidance corresponding to the identified relaxation method" is a function that provides the user with video guidance or audio guidance for carrying out the proposed relaxation method.

[1797] "Means for using smart glasses in autonomous vehicles" refers to a function that uses smart glasses installed in autonomous vehicles to display and guide information on relaxation methods through the user's gaze or voice input.

[1798] MODE FOR CARRYING OUT THE INVENTION

[1799] This invention relates to a relaxation concierge system that uses smart glasses installed in autonomous vehicles to support the user's mental health. This system receives and analyzes input from the user to suggest appropriate relaxation methods, and further identifies and presents service providers based on those methods. It also has an algorithm that receives feedback after performing a relaxation method and improves its accuracy.

[1800] Hardware Configuration

[1801] Terminal: Smart glasses installed in autonomous vehicles. A device that collects and displays data through the user's gaze and voice input.

[1802] Server: A central server that analyzes data and proposes relaxation methods. Operated as a remote cloud server.

[1803] Network: A communications network for connecting terminals and servers. It uses the Internet or a dedicated communications network.

[1804] Software Configuration

[1805] Natural Language Processing Library (NLPProcessor): Analyzes user input, extracts keywords, and identifies mental states.

[1806] Location Service API (LocationServiceAPI): Obtains the user's location information and searches for nearby relaxation providers.

[1807] Relaxation Provider API (RelaxationServiceAPI): Provides relaxation method suggestions and related provider information to the server.

[1808] Processing flow

[1809] The user inputs text or voice through the smart glasses, saying, "Recently, work has been busy and I'm feeling stressed." This input is received by the device and sent to the server via the network. The server uses an NLP processor to analyze the input and extract keywords such as "work" and "stress." This identifies the user's mental state as "caused by work-related stress."

[1810] Next, the server executes a relaxation method suggestion function, suggesting techniques such as "deep breathing," "aromatherapy," and "yoga" to the user. A video guide or audio guide corresponding to the suggested relaxation method is displayed on the smart glasses. In addition, information on nearby relaxation providers is obtained via a location information service API. For example, "aromatherapy salons" and "yoga studios" near the user's current location are displayed.

[1811] The user performs the relaxation technique and then inputs feedback through the smart glasses. This feedback is then sent back to the server for analysis. The server then uses the collected feedback to train the suggestion algorithm and improve the accuracy of future suggestions.

[1812] Specific examples

[1813] User input: "Work has been busy and stressful lately."

[1814] Suggested relaxation methods: "Deep breathing," "Aromatherapy," and "Yoga"

[1815] Suggested prompt examples:

[1816] Please suggest ways to reduce the stress you feel at work.

[1817] "Tell me how to relax"

[1818] "Find nearby relaxation facilities"

[1819] This system provides users with relaxation techniques while driving or waiting, enabling them to maintain their mental health. Feedback is also used based on the results of practicing the provided relaxation techniques, improving the accuracy of the suggestions.

[1820] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1821] Detailed explanation of the processing steps

[1822] Step 1:

[1823] Input: A user types or speaks through smart glasses, "Work has been busy and stressful lately."

[1824] Action: The device receives this input and sends it over the network to the server.

[1825] Data processing and calculation: Receives data (text or voice data) input from the smart glasses and sends it to the server as digital data.

[1826] Output: The input data is sent to the server.

[1827] Step 2:

[1828] Input: User input data sent from the device.

[1829] How it works: The server uses the NLPProcessor to parse the input data it receives and extract important keywords.

[1830] Data processing and calculation: Using natural language processing, the input data is tokenized and keywords such as "work" and "stress" are extracted.

[1831] Output: Extracted keywords.

[1832] Step 3:

[1833] Input: Extracted keywords (e.g., "work" and "stress").

[1834] Operation: The server runs an algorithm to suggest relaxation methods based on the extracted keywords.

[1835] Data processing and calculation: Keywords are compared with past databases to select the most suitable relaxation method (e.g., "deep breathing," "aromatherapy," "yoga").

[1836] Output: A list of suggested relaxation methods.

[1837] Step 4:

[1838] Input: A list of suggested relaxation methods.

[1839] How it works: The server prepares video and audio guides corresponding to relaxation methods and sends them to the smart glasses.

[1840] Data processing and calculation: Searching and selecting video and audio files corresponding to the relaxation methods.

[1841] Output: Video guide and audio guide data.

[1842] Step 5:

[1843] Input: A list of suggested relaxation methods and the user's location.

[1844] How it works: The server uses the LocationService API to search for nearby relaxation providers and sends them to the smart glasses.

[1845] Data processing and calculation: Based on the user's location information, nearby facilities that offer the proposed relaxation methods are identified and information is collected.

[1846] Output: A list of nearby relaxation providers.

[1847] Step 6:

[1848] Input: Video and audio guide data, and a list of nearby relaxation providers.

[1849] How it works: The device displays this information on the smart glasses display.

[1850] Data processing and calculation: Present data to the user visually or audibly.

[1851] Output: The user receives information through the smart glasses that encourages them to practice relaxation techniques and provides guidance on nearby facilities.

[1852] Step 7:

[1853] Input: After the user performs the relaxation technique, they enter feedback about their experience.

[1854] Operation: The terminal receives the user's feedback and transmits it to the server via the network.

[1855] Data processing and calculation: Feedback data is collected and stored in a database.

[1856] Output: User feedback is sent to the server.

[1857] Step 8:

[1858] Input: User feedback.

[1859] How it works: The server analyzes the feedback and uses it to improve the accuracy of the proposed algorithm.

[1860] Data processing and calculation: Improve the accuracy of suggestions by re-training the algorithm based on feedback data.

[1861] Output: Relaxation suggestions will be performed more accurately from the next time onwards.

[1862] This series of processes allows users to receive efficient mental care while in an autonomous vehicle.

[1863] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1864] This invention provides a system that combines an emotion engine to support users' mental health care. This system analyzes input from the user and suggests appropriate relaxation methods based on the user's emotional state. It also has the function of searching for service providers related to the suggested relaxation methods and presenting them to the user.

[1865] First, the user inputs their daily stress, anxiety, and specific worries into the device. The system then incorporates an emotion engine to recognize emotions from the user's facial expressions and voice. The device then transmits the user's text input, facial expression analysis, and voice analysis data to the server.

[1866] The server analyzes the received text data using natural language processing technology to extract key keywords. It also analyzes the user's emotions recognized by the emotion engine and compares them with the mental state extracted from the text data.

[1867] For example, consider the case where a user inputs "I've been busy at work lately and feeling stressed," and emotions of tension and anxiety are recognized from their facial expressions and voice. The server extracts the keywords "work" and "stress," and by combining these with the emotions identified by the emotion engine as "tension" and "anxiety," identifies the user's mental state as "stress, tension, and anxiety due to work."

[1868] Next, the server will list appropriate relaxation methods based on the identified mental state, such as "deep breathing," "aromatherapy," and "yoga." These relaxation methods are intended to reduce the tension and anxiety identified by the emotion engine.

[1869] The server then searches for service providers related to the selected relaxation method, identifying nearby aromatherapy salons and yoga classes based on the user's location information from affiliated databases and online resources, and transmitting information about the identified service providers to the device.

[1870] The terminal visually presents the information about the service provider to the user. For example, the terminal displays a text message such as "Click here for the nearest aromatherapy salon" and a link on the user's screen.

[1871] After a user visits a salon or class and tries out a relaxation method, the device receives feedback about the effects and impressions of the experience. For example, if a user types, "I tried aromatherapy and it helped me relax a little," the device sends this feedback to the server.

[1872] Finally, the server analyzes the received feedback and reflects the analysis results in the emotion engine, which improves the accuracy of relaxation method suggestions from the next time onwards and makes suggestions that are more suited to the user's individual emotional state.

[1873] As a concrete example of this system, if a user inputs "I can't sleep at night and I'm frustrated lately," and the server recognizes fatigue from the user's facial expression and voice, it will combine keywords such as "can't sleep" and "frustrated" with "fatigue" to identify the symptom as "fatigue and frustration due to lack of sleep." In this case, the server will suggest methods such as "aromatherapy" and "relaxation music," and search for and display nearby related service providers.

[1874] In this way, by combining an emotion engine, the present invention provides a system that proposes more accurate relaxation methods based on the user's emotional state and quickly searches for and presents service providers, allowing users to easily find the relaxation method that best suits them and effectively relieve stress.

[1875] The processing flow will be explained below.

[1876] Step 1:

[1877] The user inputs information about their stress and anxiety, for example, "Work has been busy lately and I'm feeling stressed."

[1878] Step 2:

[1879] The device receives the user's input as text data, and also acquires the user's facial expression and voice data.

[1880] Step 3:

[1881] The terminal transmits the received text data, facial expression data, and voice data to the server.

[1882] Step 4:

[1883] The server receives the data sent from the terminal.

[1884] Step 5:

[1885] The server uses natural language processing technology to analyze the text data and extract key keywords. In this case, the keywords extracted are "work" and "stress."

[1886] Step 6:

[1887] The server uses an emotion engine to analyze the received facial expression and voice data and recognize the user's emotions, such as "tension" or "anxiety."

[1888] Step 7:

[1889] The server compares the keywords extracted from the text data with the emotions recognized by the emotion engine to identify the user's mental state. Specifically, it identifies "stress, tension, and anxiety due to work."

[1890] Step 8:

[1891] Based on the identified mental state, the server will then list appropriate relaxation methods, such as "deep breathing," "aromatherapy," and "yoga."

[1892] Step 9:

[1893] The server searches for service providers related to the selected relaxation method, searching a database for nearby aromatherapy salons and yoga classes based on the user's location information.

[1894] Step 10:

[1895] The server prepares information about the service providers obtained as a result of the search and transmits it to the terminal.

[1896] Step 11:

[1897] The terminal receives the information about the service provider sent from the server.

[1898] Step 12:

[1899] The terminal visually displays the received information about the service provider to the user, providing a text such as "Click here for nearby aromatherapy salons" and a link.

[1900] Step 13:

[1901] The user can then take action based on the displayed information about relaxation methods and service providers, for example, visiting a nearby aromatherapy salon.

[1902] Step 14:

[1903] After performing the relaxation method, the user inputs the results as feedback into the terminal, for example, "I tried aromatherapy and it helped me relax a little."

[1904] Step 15:

[1905] The terminal receives feedback from the user and transmits it to the server.

[1906] Step 16:

[1907] The server receives the feedback sent from the terminal.

[1908] Step 17:

[1909] The server analyzes the feedback content and evaluates the effectiveness of the relaxation method.

[1910] Step 18:

[1911] The server reflects the analysis results in the emotion engine, improving the accuracy of relaxation method suggestions from the next time onwards.

[1912] This series of steps allows users to easily find the relaxation method that best suits them and provides effective stress relief measures based on their emotions and mental state.

[1913] Example 2

[1914] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1915] In modern society, users experience a great deal of stress and anxiety due to their daily lives and work. To properly care for these mental health issues, a system is needed that can accurately assess the user's condition and suggest optimal relaxation methods. It is also necessary to analyze the user's emotional state in real time and respond quickly according to that state. Furthermore, it is necessary to immediately provide information on service providers related to the suggested relaxation methods, thereby reducing the burden on the user. Conventional systems have had difficulty meeting these requirements, so providing customized mental health care to users has become a challenge.

[1916] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1917] In this invention, the server includes means for receiving input from a user, means for analyzing the received input and identifying the user's mental state, means for suggesting an appropriate relaxation method based on the analysis results, means for searching for service providers related to the suggested relaxation method, means for presenting information about the service providers to the user, means for collecting the user's facial expression data and voice data and analyzing them using an emotion engine, means for receiving and analyzing feedback from the user, and means for improving the accuracy of suggested relaxation methods based on the feedback. This enables accurate suggestions of relaxation methods based on the user's individual mental state and rapid search and presentation of related service providers.

[1918] "User input" is information provided by a user to a terminal through text, voice, or facial expressions.

[1919] "Mental state" refers to the mental and psychological state of the user, such as stress, anxiety, and worries.

[1920] "Relaxation methods" are methods such as deep breathing, aromatherapy, and yoga that are proposed as means for improving the user's mental state.

[1921] "Service provider" refers to a person or facility that provides services related to relaxation methods.

[1922] "Suggestion accuracy" indicates the degree of accuracy and relevance when suggesting relaxation methods appropriate for the user's mental state.

[1923] An "emotion engine" is a technology or algorithm that analyzes a user's facial expressions and voice data to identify their emotional state.

[1924] "Natural language processing technology" is a technology for analyzing user input text and understanding its meaning.

[1925] "Feedback" refers to information about impressions and effects provided by users after using the proposed relaxation method or related service.

[1926] "Location information" is information that indicates the geographical location where the user currently resides.

[1927] This invention is a system that supports users' mental health care and uses technology that combines an emotion engine. Users input their daily stress, anxiety, and specific worries into a terminal, and facial expression and voice data are also collected. This input data is sent to a server and analyzed.

[1928] Hardware and software used

[1929] This system uses the following hardware and software.

[1930] Terminal: The device through which the user provides input (e.g., smartphone, PC).

[1931] Emotion engine: Technology for analyzing the user's facial expressions and voice (OpenCV, IBM Watson, Google Cloud Speech-to-Text, etc.).

[1932] Server: A computer system for analyzing and processing data.

[1933] Natural language processing technologies: Technologies for analyzing user text input (NLTK library, generative AI models such as GPT-3).

[1934] System operation explanation

[1935] 1. User Input

[1936] Users use the device to input their daily stress, anxiety, and worries in text format, while facial expression and voice data are also collected at the same time.

[1937] Example: A user types, "I've been busy at work lately and feeling stressed," and their facial expressions and voice are also recorded.

[1938] 2. Sending data to the server

[1939] The terminal transmits the input text data, facial expression data, and voice data to the server.

[1940] Technical example: A smartphone or PC sends data to a server as an HTTP request.

[1941] 3. Text Data Analysis

[1942] The server uses natural language processing technology to analyze the received text data and extract key keywords.

[1943] Technologies used: Python's NLTK library and generative AI models (such as GPT-3).

[1944] Example: Extract the keywords "work" and "stress" from the analysis results of the text "I've been busy at work lately and feeling stressed."

[1945] 4. Emotion Data Analysis

[1946] The server uses an emotion engine to analyze facial and voice data to identify the user's emotional state (e.g., tension, anxiety).

[1947] Technologies used: OpenCV for facial expression recognition, IBM Watson and Google Cloud Speech-to-Text for voice emotion recognition.

[1948] For example, "tension" can be recognized from facial expressions, and "anxiety" can be recognized from voice.

[1949] 5. Relaxation suggestions

[1950] Based on the analysis results, the server identifies the user's mental state and suggests appropriate relaxation methods (deep breathing, aromatherapy, yoga, etc.).

[1951] Example: Suggest "deep breathing," "aromatherapy," and "yoga" to combat "stress, tension, and anxiety caused by work."

[1952] 6. Search for a service provider

[1953] The server searches affiliated databases and online resources for service providers related to relaxation techniques.

[1954] The search for service providers is carried out based on the user's location information.

[1955] For example: Identify nearby aromatherapy salons or yoga classes.

[1956] 7. Presentation of Service Provider Information

[1957] The terminal visually presents the search results to the user and provides information on nearby service providers.

[1958] Example: Displaying text and a link such as "Click here for aromatherapy salons near you" on the user screen.

[1959] 8. User Feedback

[1960] After actually using the service, the user inputs the results and impressions into the terminal as feedback.

[1961] Example: User types, "I tried aromatherapy and it helped me relax a bit."

[1962] 9. Incorporating Feedback

[1963] The server analyzes the received feedback and reflects it in the emotion engine, thereby improving the accuracy of relaxation method suggestions from the next time onwards.

[1964] Technical example: Integrating feedback into the system by retraining machine learning models.

[1965] Through the above series of steps, the present invention provides a system that effectively supports users' mental health care. By combining specific technologies and operations, it is possible to accurately suggest relaxation methods based on the user's emotional state and quickly search for and present service providers.

[1966] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1967] Step 1:

[1968] User Input and Data Capture

[1969] Users use a device to input their daily stress, anxiety, and worries in text format. The system also collects facial and voice data. The input data includes text, facial images, and voice files.

[1970] Input: Text data, facial expression data, voice data

[1971] How it works: The user types, "I've been busy at work lately and feeling stressed," and their facial expressions and voice are also recorded. The device's camera and microphone capture the user's facial expressions and voice in real time.

[1972] Output: Input data (text, facial expressions, voice)

[1973] Step 2:

[1974] Sending data to the server

[1975] The terminal transmits the collected text data, facial expression data, and voice data to the server.

[1976] Input: Input data (text, facial expressions, voice)

[1977] How it works: Your smartphone or PC sends data to the server as an HTTP request, which is then formatted and encoded.

[1978] Output: Received data on the server (text, facial expressions, voice)

[1979] Step 3:

[1980] Text data analysis

[1981] The server analyzes the received text data using natural language processing technology and extracts key keywords.

[1982] Input: Received text data on the server

[1983] How it works: It uses Python's NLTK library and generative AI models (such as GPT-3) to extract important keywords from text.

[1984] Output: Extracted keywords (e.g., "work" and "stress")

[1985] Step 4:

[1986] Emotional Data Analysis

[1987] The server uses an emotion engine to analyze facial expression and voice data to identify the user's emotional state.

[1988] Input: Received facial expression data and voice data on the server

[1989] Operation: Classifies emotional states using OpenCV for facial expression recognition and IBM Watson or Google Cloud Speech-to-Text for voice emotion recognition. Example: "tension" or "anxiety."

[1990] Output: Identified emotion (e.g., "tension" or "anxiety")

[1991] Step 5:

[1992] Suggestions for relaxation methods

[1993] The server combines the analyzed keywords with the user's emotional state to identify the user's mental state and list appropriate relaxation methods.

[1994] Input: extracted keywords, identified sentiments

[1995] How it works: It detects the user's mental state and runs an algorithm that selects relaxation methods (e.g., "deep breathing," "aromatherapy," "yoga") based on that.

[1996] Output: A list of suggested relaxation techniques (e.g., "deep breathing," "aromatherapy," "yoga")

[1997] Step 6:

[1998] Search for a service provider

[1999] The server searches affiliated databases and online resources for service providers related to relaxation methods, and identifies the service provider based on the user's location information.

[2000] Input: List of relaxation methods, user location

[2001] How it works: Searches for nearby service providers (e.g., aromatherapy salons or yoga classes) through affiliated databases and APIs, and obtains rankings and reviews.

[2002] Output: List of relevant service providers with details

[2003] Step 7:

[2004] Present service provider information

[2005] The terminal visually presents information about service providers to the user based on the search results.

[2006] Input: List of relevant service providers and details

[2007] Operation: Displays information about service providers on the user interface. For example, it displays text such as "Click here for nearby aromatherapy salons," links, and map information on the screen.

[2008] Output: Service provider information presented to the user

[2009] Step 8:

[2010] User Feedback

[2011] After actually using the service, the user inputs feedback about its effectiveness and impressions into the terminal.

[2012] Input: User feedback data (text format)

[2013] What it does: Displays a feedback form and allows the user to fill it in. For example, the user might type, "I tried aromatherapy and it helped me relax a bit."

[2014] Output: Feedback data

[2015] Step 9:

[2016] Reflecting feedback

[2017] The server analyzes the received feedback and reflects it in the emotion engine, thereby improving the accuracy of relaxation method suggestions from the next time onwards.

[2018] Input: User feedback data

[2019] How it works: The feedback data is analyzed and used as data to retrain the emotion engine model, thereby improving the accuracy of the proposed algorithm.

[2020] Output: Emotion engine and proposed algorithm with improved accuracy

[2021] (Application example 2)

[2022] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2023] Conventional mental healthcare systems estimate a user's mental state based on text input and suggest relaxation methods. However, in order to recognize the user's emotional state with greater accuracy, it is necessary to incorporate facial expression and voice analysis. Furthermore, systems lack the ability to not only suggest appropriate relaxation methods but also to quickly search for and present relevant service providers. Furthermore, systems lack the ability to identify service providers based on the user's location information or to incorporate user feedback to improve the accuracy of future suggestions.

[2024] The identification process performed by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving input from the user, means for analyzing the received input and identifying the user's mental state, means for recognizing the user's emotional state through facial expression analysis and voice analysis, means for suggesting an appropriate relaxation method based on the analysis results, means for searching for service providers related to the suggested relaxation method, means for presenting information about the service providers to the user, means for receiving and analyzing feedback from the user, and means for improving the accuracy of relaxation method suggestions based on the feedback. This makes it possible to recognize the user's emotional state with higher accuracy and quickly identify and present appropriate relaxation methods and related service providers. Furthermore, using the feedback improves the accuracy of suggestions from the next time onwards.

[2025] "Means for receiving input from a user" refers to a device that allows a user to input text or voice and software for processing the input.

[2026] The "means for analyzing the received input content and identifying the user's mental state" refers to an algorithm and processing device that analyzes the text and voice data entered by the user and infers the user's emotions and psychological state from the content.

[2027] "Means for recognizing a user's emotional state through facial expression analysis and voice analysis" refers to technology that analyzes a user's facial expressions and tone of voice and recognizes their emotional state from the results.

[2028] The "means for suggesting an appropriate relaxation method based on the analysis results" refers to an algorithm and processing device for selecting and suggesting a relaxation method based on the analysis results of the user's mental and emotional state.

[2029] "Means for searching for service providers related to the proposed relaxation method" refers to a database and search engine for searching and identifying businesses and facilities that provide services according to the proposed relaxation method.

[2030] The "means for presenting information of service providers to a user" refers to a user interface and software for displaying the searched information of service providers on a user's device.

[2031] The "means for receiving and analyzing feedback from the user" refers to a means for collecting and analyzing the user's impressions and experiences after implementing the relaxation method.

[2032] The "means for improving the accuracy of relaxation method suggestions based on feedback" refers to an algorithm and processing device that uses the results of analyzing user feedback to improve the accuracy of relaxation method suggestions from the next time onwards.

[2033] The "means for identifying nearby service providers based on the user's location information" refers to a geographic information system and search engine for identifying nearby relevant service providers based on the user's current location information.

[2034] "Means for analyzing user input using natural language processing technology" refers to natural language processing algorithms and software that analyze text data entered by users and extract key keywords and emotions.

[2035] The present invention is a system for supporting a user's mental health care. This system receives input data from the user, analyzes it, and suggests appropriate relaxation methods. It also has the function of searching for service providers related to the suggested relaxation methods and presenting them to the user. The system also receives feedback and improves the accuracy of relaxation method suggestions from the next time onwards.

[2036] Hardware and Software Examples

[2037] 1. Hardware:

[2038] Smartphones (iOS and Android devices)

[2039] server

[2040] 2. Software:

[2041] Natural Language Processing: Google Cloud Natural Language API

[2042] Facial expression analysis: Amazon Rekognition

[2043] Speech analysis: Google Cloud Speech-to-Text API

[2044] Location search: Google Maps API

[2045] Data processing and calculation flow

[2046] 1. Receiving user input:

[2047] Users use a smartphone application to input their stress and anxiety levels via text or voice, while facial expression data is also collected via the camera.

[2048] 2. Analysis of input data:

[2049] The text data is analyzed using the Google Cloud Natural Language API to extract key keywords and the user's mental state.

[2050] Facial expression data is analyzed using Amazon Rekognition to identify the user's emotions.

[2051] The audio data is converted to text using the Google Cloud Speech-to-Text API, and then sentiment analysis is performed using the Google Cloud Natural Language API.

[2052] 3. Relaxation suggestions:

[2053] Based on the analysis results, a database is consulted to suggest appropriate relaxation techniques (e.g., deep breathing, aromatherapy, yoga).

[2054] 4. Search and present service providers:

[2055] Service providers related to the proposed relaxation method (e.g., aromatherapy salons, yoga classes) are searched for using the user's location information using the Google Maps API and presented to the user.

[2056] 5. Feedback Collection and Analysis:

[2057] After trying out a relaxation technique, users can enter their impressions and feedback through the application. This feedback is analyzed by the server and used to improve the accuracy of future suggestions.

[2058] Specific examples

[2059] 1. User Input:

[2060] The user types, "Work has been busy and stressful lately."

[2061] Facial expression analysis recognizes the emotion of "tension" from the user's face.

[2062] Voice analysis identifies the emotion of "anxiety" from voice.

[2063] 2. Relaxation suggestions:

[2064] The server suggests "deep breathing," "aromatherapy," and "yoga" as suitable relaxation methods.

[2065] 3. Service Provider Presentation:

[2066] The text and link "Here are the nearest aromatherapy salons" will appear on the app screen.

[2067] Prompt Sentence Examples

[2068] User input: "Work has been busy and stressful lately."

[2069] Recognized emotional state: Facial expression analysis "tension", voice analysis "anxiety"

[2070] Extracted keywords: "work", "stress"

[2071] Suggested relaxation methods: "Deep breathing", "Aromatherapy", "Yoga"

[2072] Search for nearby service providers: Google Maps API

[2073] This allows for more accurate recognition of the user's emotional state, enabling the system to suggest appropriate relaxation methods and quickly search for and present relevant service providers. Furthermore, by using user feedback, the system can improve the accuracy of suggestions from the next time onwards, providing a more personalized experience.

[2074] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2075] Step 1:

[2076] A user inputs text or voice using a smartphone application.

[2077] Input: User text input, voice input, and facial expression data

[2078] Specific behavior:

[2079] The user inputs the stress or anxiety they are experiencing through text or voice.

[2080] Facial expression data is collected via a smartphone camera.

[2081] Output: Input data (text, voice, facial expression)

[2082] Step 2:

[2083] The terminal transmits the collected input data to the server.

[2084] Input: Input data (text, voice, facial expression)

[2085] Specific behavior:

[2086] The terminal transmits the text data as is, the voice data as a voice file, and the facial expression data as an image to the server.

[2087] Output: Input data sent to the server

[2088] Step 3:

[2089] The server analyzes the received text data using the Google Cloud Natural Language API to extract key keywords and mental states.

[2090] Input: Text data

[2091] Specific behavior:

[2092] The server calls the Google Cloud Natural Language API to parse the text data.

[2093] Extract key keywords (e.g., "work," "stress") and the user's mental state.

[2094] Output: Key keywords and mental states

[2095] Step 4:

[2096] The server analyzes the facial expression data using Amazon Rekognition and recognizes the user's emotions.

[2097] Input: facial expression data (image)

[2098] Specific behavior:

[2099] The server calls Amazon Rekognition to analyze the facial expression data.

[2100] Recognize the user's emotional state (e.g., "tension") from facial expressions.

[2101] Output: Perceived emotional state

[2102] Step 5:

[2103] The server converts the voice data into text using the Google Cloud Speech-to-Text API, and then analyzes the sentiment using the Google Cloud Natural Language API.

[2104] Input: Audio data

[2105] Specific behavior:

[2106] The server calls the Google Cloud Speech-to-Text API to convert the audio data into text.

[2107] The converted text data is then analyzed again using the Google Cloud Natural Language API to identify sentiment.

[2108] Output: Perceived emotional state

[2109] Step 6:

[2110] The server refers to a database to suggest appropriate relaxation methods based on the analysis results.

[2111] Input: Primary keywords, perceived emotional state

[2112] Specific behavior:

[2113] Based on the analysis results, the most suitable relaxation method for the user (e.g., "deep breathing," "aromatherapy," "yoga") is selected from a database.

[2114] Output: Proposed relaxation method

[2115] Step 7:

[2116] The server uses the Google Maps API to search for nearby service providers related to the proposed relaxation method.

[2117] Input: Proposed relaxation method, user location

[2118] Specific behavior:

[2119] Call the Google Maps API to search for service providers based on the user's location information.

[2120] Identify service providers suitable for the proposed relaxation method.

[2121] Output: Service provider information

[2122] Step 8:

[2123] The terminal presents information about the service provider to the user.

[2124] Input: Service provider information

[2125] Specific behavior:

[2126] Display information about the service provider (e.g., "Click here for the nearest aromatherapy salon") in text and link format on the user's app screen.

[2127] Output: Information presented to the user

[2128] Step 9:

[2129] After the user tries the relaxation techniques, they enter feedback through the application.

[2130] Input: User feedback (text)

[2131] Specific behavior:

[2132] The user inputs the effects and impressions of the relaxation method in text format.

[2133] Output: Input feedback

[2134] Step 10:

[2135] The terminal transmits the feedback from the user to the server.

[2136] Input: User feedback

[2137] Specific behavior:

[2138] The terminal transmits the feedback text data to the server.

[2139] Output: Feedback sent to the server

[2140] Step 11:

[2141] The server analyzes the received feedback and updates the database to improve the accuracy of relaxation method suggestions from the next time onwards.

[2142] Input: Feedback data

[2143] Specific behavior:

[2144] The feedback data will be analyzed and the results will be used to update the database to improve the accuracy of relaxation method suggestions.

[2145] Output: Updated database

[2146] The above processing steps enable highly accurate recognition of the user's emotional state, suggesting appropriate relaxation methods, and quickly searching and presenting relevant service providers. Furthermore, by using feedback, the accuracy of suggestions can be improved in future visits, providing a more personalized experience.

[2147] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2148] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2149] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2150] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2151] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2152] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2153] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2154] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2155] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2156] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2157] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2158] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2159] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2160] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2161] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2162] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2163] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2164] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2165] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2166] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2167] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2168] The following is further disclosed regarding the above embodiment.

[2169] (Claim 1)

[2170] means for receiving input from a user;

[2171] means for analyzing the received input and determining the mental state of the user;

[2172] A means for suggesting appropriate relaxation methods based on the analysis results;

[2173] A means for searching for service providers related to the proposed relaxation method;

[2174] means for presenting information about service providers to a user;

[2175] means for receiving and analyzing feedback from users;

[2176] A method for improving the accuracy of relaxation method suggestions based on feedback;

[2177] A system including:

[2178] (Claim 2)

[2179] 10. The system of claim 1, further comprising means for identifying nearby service providers based on location information of the user.

[2180] (Claim 3)

[2181] 10. The system of claim 1, further comprising means for analyzing user input using natural language processing techniques.

[2182] "Example 1"

[2183] (Claim 1)

[2184] means for receiving input from a user;

[2185] means for tokenizing the received input and extracting keywords to identify the user's mental state;

[2186] Based on the extracted keywords and analysis results, a method is provided to list and suggest appropriate relaxation methods.

[2187] means for searching databases and online resources for service providers related to the proposed relaxation method;

[2188] means for visually presenting information about service providers to a user;

[2189] means for receiving and analyzing feedback from users;

[2190] A method for improving the accuracy of relaxation method suggestions using a machine learning model based on the feedback; and

[2191] A system including:

[2192] (Claim 2)

[2193] 10. The system of claim 1, further comprising means for identifying nearby service providers based on location information of the user.

[2194] (Claim 3)

[2195] 10. The system of claim 1, further comprising means for tokenizing the user's input and extracting keywords using natural language processing techniques.

[2196] "Application Example 1"

[2197] Restated Claims

[2198] (Claim 1)

[2199] means for receiving input from a user;

[2200] means for analyzing the received input and determining the mental state of the user;

[2201] A means for suggesting appropriate relaxation methods based on the analysis results;

[2202] a means for searching for providers related to the proposed relaxation method;

[2203] means for presenting information about the provider to a user;

[2204] means for receiving and analyzing feedback from users;

[2205] A method for improving the accuracy of relaxation method suggestions based on feedback;

[2206] means for providing a video or audio guide corresponding to the identified relaxation technique;

[2207] A method for using smart glasses in an autonomous vehicle to support the mental health of the driver based on a specified relaxation technique;

[2208] A system including:

[2209] (Claim 2)

[2210] 10. The system of claim 1, further comprising means for identifying nearby providers based on location information of the user.

[2211] (Claim 3)

[2212] 10. The system of claim 1, further comprising means for analyzing user input using natural language processing techniques.

[2213] "Example 2: Combining Emotion Engines"

[2214] (Claim 1)

[2215] means for receiving input from a user;

[2216] means for analyzing the received input and determining the mental state of the user;

[2217] A means for suggesting appropriate relaxation methods based on the analysis results;

[2218] A means for searching for service providers related to the proposed relaxation method;

[2219] means for presenting information about service providers to a user;

[2220] means for collecting facial expression data and voice data of a user and analyzing the data using an emotion engine;

[2221] means for receiving and analyzing feedback from users;

[2222] A method for improving the accuracy of relaxation method suggestions based on feedback;

[2223] A system including:

[2224] (Claim 2)

[2225] 10. The system of claim 1, further comprising means for searching and identifying nearby service providers based on the user's location information.

[2226] (Claim 3)

[2227] 10. The system of claim 1, further comprising means for analyzing user input using natural language processing techniques.

[2228] "Application example 2 when combining emotion engines"

[2229] (Claim 1)

[2230] means for receiving input from a user;

[2231] means for analyzing the received input and determining the mental state of the user;

[2232] means for recognizing the emotional state of a user through facial expression and voice analysis;

[2233] A means for suggesting appropriate relaxation methods based on the analysis results;

[2234] A means for searching for service providers related to the proposed relaxation method;

[2235] means for presenting information about service providers to a user;

[2236] means for receiving and analyzing feedback from users;

[2237] A method for improving the accuracy of relaxation method suggestions based on feedback;

[2238] A system including:

[2239] (Claim 2)

[2240] 10. The system of claim 1, further comprising means for identifying nearby service providers based on location information of the user.

[2241] (Claim 3)

[2242] 10. The system of claim 1, further comprising means for analyzing user input using natural language processing techniques. [Explanation of symbols]

[2243] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving input from a user; means for analyzing the received input and determining the mental state of the user; A means for suggesting appropriate relaxation methods based on the analysis results; A means for searching for service providers related to the proposed relaxation method; means for presenting information about service providers to a user; means for receiving and analyzing feedback from users; A method for improving the accuracy of relaxation method suggestions based on feedback; A system including:

2. The system of claim 1 , further comprising means for identifying nearby service providers based on location information of the user.

3. 10. The system of claim 1, further comprising means for analyzing user input using natural language processing techniques.

Images