system

Abstract

We provide the system. [Solution] A means of analyzing user input using natural language processing to identify the user's intent, A means of automatically collecting and analyzing relevant data from the internet, A means of generating optimal suggestions for users using collected data, A means of sending the generated proposal to the user's terminal and presenting it to the user, A means of collecting and learning data to improve the accuracy of suggestions based on user feedback, A system that includes means of providing users with the most suitable payment options and discount information in real time.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Modern consumers have to individually investigate a vast amount of information related to daily shopping and going out, which causes a waste of time and labor. Also, due to the dispersion of information, it is difficult to make an optimal choice. There is a demand for a system that can improve such a situation and enable consumers to efficiently acquire information and make an optimal decision.

Means for Solving the Problems

[0005] This invention includes means for analyzing user input using natural language processing to identify the user's intent, and means for automatically collecting and analyzing relational data on the internet. It also includes means for generating optimal suggestions for the user based on the collected data and transmitting them to the user's terminal. Furthermore, it includes means for collecting and learning data to improve the accuracy of suggestions based on user feedback.

[0006] "User input" refers to information provided by system users to specify their purpose or preferences.

[0007] "Natural language processing" is a technology that enables computers to understand and process the language that humans use on a daily basis.

[0008] The "Internet" is an information and communication network that connects computer networks around the world.

[0009] "Relational data" refers to information or data that is related to a specific purpose or condition.

[0010] "Analysis" is the process of breaking down data and information into individual elements to understand their content.

[0011] A "suggestion" is information or advice presented to recommend a particular option.

[0012] A "user terminal" refers to a computer or portable device used by a user to access or input information.

[0013] "Feedback" refers to the evaluations and opinions that users provide regarding the system's suggestions and results.

[0014] "Suggestion accuracy" refers to the degree to which a system provides suggestions that closely match the user's expectations and needs.

[0015] "Learning" is the process by which a system improves its performance based on past data and experience.

Brief Description of Drawings

[0016] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Modes for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention is a system that supports users in efficiently acquiring information and making optimal decisions in their daily lives. The system operates with its components working together as follows:

[0038] 1. Processing user input

[0039] Users input their wishes and objectives in natural language via their devices. For example, "I want to have a party this weekend, so I'd like information on ingredients and venues."

[0040] 2. Analysis using natural language processing

[0041] The server receives user input sent from the terminal and analyzes it using natural language processing technology. This analysis identifies the user's intent and the information they need.

[0042] 3. Collection of related data

[0043] The server collects necessary data from the internet based on the analysis results. Specifically, it acquires website data to obtain information about products and services that users are looking for, as well as the best prices.

[0044] 4. Proposal generation

[0045] The server analyzes the collected data and generates suggestions that best meet the user's needs. This includes price comparisons, location guidance, and coupon information.

[0046] 5. Presenting information to the user

[0047] The terminal displays suggested information received from the server to the user. The user can also ask additional questions using the voice assistant.

[0048] 6. Gathering and Learning from Feedback

[0049] The user acts based on the presented suggestion. The device records the result and sends it to the server. The server learns from this feedback and uses it as data to improve future suggestions.

[0050] As a concrete example, if a user is planning to purchase daily necessities, the system can be used as follows: When the user inputs "I want to buy detergent and shampoo by this weekend," the server analyzes the request, compares information on nearby stores and online prices, and suggests the best purchase options. Furthermore, the server can refer to the user's past purchase history and recommend their favorite brands. The terminal presents this information to the user, who then makes a purchase based on the suggestions. The user's purchase behavior is sent to the server as feedback and used to improve the accuracy of future suggestions.

[0051] This configuration allows users to obtain information efficiently and without hassle, enabling them to make optimal choices.

[0052] The following describes the processing flow.

[0053] Step 1:

[0054] The user inputs their wishes and requests related to their daily life into the terminal using natural language. This input serves as the initial trigger for this system.

[0055] Step 2:

[0056] The terminal transmits user input as digital data to the server. This data is necessary to analyze the user's intent.

[0057] Step 3:

[0058] The server uses natural language processing techniques to analyze user input. This analysis specifically identifies the user's requests and objectives, and determines the type of information needed.

[0059] Step 4:

[0060] The server collects relevant data from the internet based on the analysis results. This data collection is performed through API access and web scraping.

[0061] Step 5:

[0062] The server uses AI technology to analyze the collected data and generate optimal recommendations for the user. This analysis includes price comparisons, quality evaluations, and user reviews.

[0063] Step 6:

[0064] The server sends the generated suggestion to the terminal. This suggestion is a structured version of the information the user is looking for.

[0065] Step 7:

[0066] The terminal displays suggestions from the server to the user via a user interface. The user can then ask more detailed questions by activating the voice assistant.

[0067] Step 8:

[0068] The user makes decisions based on the suggestions presented on the device. For example, they might choose to make a purchase at a suggested store.

[0069] Step 9:

[0070] After a user's action, the device records the result and sends the user's feedback to the server.

[0071] Step 10:

[0072] The server uses feedback to update the AI ​​model and learn to improve the accuracy of future suggestions.

[0073] (Example 1)

[0074] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0075] In modern society, there are limited means to efficiently gather the information users need and support them in making optimal decisions. In particular, in situations where quick and appropriate information acquisition is required in daily activities, users find it difficult to discern the best option from many choices. Furthermore, in an information-overload environment, mechanisms for providing personalized suggestions based on individual needs and preferences are not yet fully established.

[0076] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0077] In this invention, the server includes means for analyzing natural language input received from a user via an information device to identify the user's intent, means for automatically collecting and analyzing relevant information from a database on the Internet using an information processing device, and means for generating suggestions suitable for the user's request based on the collected information. This makes it possible to obtain optimal information and provide suggestions based on the user's intent.

[0078] "Information equipment" is a general term for terminals and devices that users use to input information.

[0079] "Natural language input" is a method by which users provide information to information devices using natural language and sentences.

[0080] An "information processing device" is a computing device used to collect, analyze, and process digital data.

[0081] A "database" is a structured collection of information used to systematically store and access necessary information.

[0082] "Relevant information" refers to data and facts related to the user's intentions and requests.

[0083] "Proposal" means presenting users with convenient and optimal options or actions based on the information collected.

[0084] "Feedback information" refers to information about the results and reactions related to user actions and choices.

[0085] "Personalized suggestions" refer to suggestions that are specially customized to match the user's past behavior history and preferences.

[0086] This invention is a system that supports the efficient acquisition of information and the optimization of decision-making in the user's daily life. This system is implemented using information equipment, servers, and related software technologies.

[0087] First, the user inputs their purpose or request in natural language via an information device. For example, they might input a request such as, "I'm looking for a place to meet up with friends this weekend, and I'd like to know the best restaurant." This input is received by the terminal, and the user's response is sent to the server as a prompt. An example of such a prompt would be, "Please recommend a restaurant for the weekend."

[0088] Next, the server utilizes a generative AI model and natural language processing techniques to analyze user input. Through this analysis, the server identifies the user's intent and, based on that information, collects relevant data from the internet. This data collection can be done using database queries or web scraping techniques. Specifically, AI technologies known as generative AI models and web scraping tools are used.

[0089] The server then processes the collected information using advanced data analysis techniques to generate optimal suggestions for the user. For example, based on the collected restaurant information, it provides personalized suggestions based on the user's preferences and past reviews. These suggestions include price comparisons and restaurant location information.

[0090] Finally, the terminal displays the suggestions received from the server to the user. The user makes a decision based on this information and sends the result and feedback back to the server via the terminal. The server analyzes this feedback information and uses it to improve the accuracy of the suggestions. This adjusts future suggestions to be more accurate and user-oriented.

[0091] This embodiment allows users to intuitively and efficiently obtain the necessary information and make decisions without having to perform cumbersome information searches.

[0092] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0093] Step 1:

[0094] The user uses an information device to input specific purposes or wishes in natural language. The input text is received on the user's device, and a prompt message such as "Please recommend some restaurants for the weekend" is generated. The input natural language text becomes the basic data for subsequent processing.

[0095] Step 2:

[0096] The terminal sends user input to the server. During this process, the input data is securely encrypted using the SSL / TLS protocol and transmitted. The transmitted prompt message is awaiting analysis by the server.

[0097] Step 3:

[0098] The server analyzes the received natural language input using a generative AI model. Specifically, it uses natural language processing techniques to identify the intent of the input sentence and extract keywords. The input is a prompt sentence, and the output is a set of analyzed keywords and themes (e.g., "restaurant," "weekend," etc.).

[0099] Step 4:

[0100] The server collects relevant data from the internet based on the analyzed keywords. Here, web scraping tools are used to gather specific store information, price information, and other data from publicly available information and databases on the internet. The input is keywords, and the output is a collection of relevant data.

[0101] Step 5:

[0102] The server processes the collected data using data analysis techniques to generate personalized recommendations for the user. For example, it compares price information from a database to suggest the most cost-effective restaurant. It also includes personalized advice based on past user preferences. The input is the collected raw data, and the output is specific recommendation information.

[0103] Step 6:

[0104] The proposed information is sent from the server to the terminal. This transmission involves formatting the data (e.g., JSON format) to ensure it is suitable for display. The input is the proposed information, and the output is the data sent to the user's terminal.

[0105] Step 7:

[0106] The device presents the user with appropriately formatted suggestion information. The user can view this information and, if necessary, request further details using the voice assistant or touch interface. The input is formatted data, and the output is visual or audible feedback to the user.

[0107] Step 8:

[0108] Users make decisions based on the information presented and provide feedback on the results and their impressions through their device. This feedback is used to improve future proposals. The input is the user's experience and choices, and the output is feedback data.

[0109] Step 9:

[0110] The server receives feedback data and uses machine learning techniques to improve the system's proposed algorithm. This process enables more accurate personalized recommendations for subsequent uses. The input is feedback data, and the output is the improved algorithm model.

[0111] (Application Example 1)

[0112] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0113] When users choose products and services in their daily lives, it is difficult for them to quickly and accurately find the best option from a vast amount of information. Furthermore, effectively utilizing payment options and discount information is also challenging. This complicates the user's purchasing experience and hinders optimal decision-making.

[0114] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0115] In this invention, the server includes means for analyzing user input using natural language processing to identify the user's intent, means for generating optimal suggestions for the user using the collected data, and means for providing the user with optimal payment options and discount information in real time. This enables users to efficiently obtain relevant information and quickly make optimal choices in their daily purchasing activities.

[0116] "User input" refers to the act of a user sending information to a system via a terminal, and the content of that information, including natural language and instructions.

[0117] "Natural language processing" is a technology that uses computers to analyze human language and understand user intent and information.

[0118] "Automatically collecting relevant data from the internet" refers to the process of automatically searching for and retrieving relevant information that exists on a network.

[0119] "Generating suggestions" is the act of organizing and presenting the most suitable options and information for the user based on collected data.

[0120] "User terminal" refers to a digital device used by a user to receive information, and includes smartphones, computers, and other similar devices.

[0121] "Improving the accuracy of suggestions based on feedback" is a process of analyzing user responses and results to improve the quality of information provided in the future.

[0122] "Payment options" refer to the payment methods and conditions available when purchasing goods or services.

[0123] "Discount information" refers to information about price reductions or benefits that apply under specific conditions.

[0124] The system for realizing this invention consists of a user-owned digital device (such as a smartphone or smart glasses) and a server connected via the internet. The server receives user input, performs natural language processing, and analyzes the user's intentions and requests. This analysis uses natural language processing libraries (e.g., spaCy, Google® NLP API).

[0125] The server automatically collects information on relevant products and services from the internet based on the analysis results. Web scraping tools (e.g., BeautifulSoup, Selenium) are used for this information collection. The collected data is then used to generate suggestions best suited to the user's needs. Generative AI models may be used to personalize these suggestions.

[0126] The user's device displays suggested information received from the server via voice and visuals. The user reviews this information and provides additional instructions as needed, such as through voice input. Furthermore, payment options and discount information related to the products and services the user is considering are also provided in real time.

[0127] As a concrete example, consider a scenario where a user is planning their grocery shopping trip using smart glasses. If the user voice-inputs, "I want to know what the cheapest detergent is right now," the system analyzes the information and immediately displays the latest price information and available discounts.

[0128] By utilizing generative AI models in this process, it becomes possible to provide optimal suggestions to users quickly and with high accuracy, improving the user's purchasing experience. It also accepts prompt messages such as, "I want to buy the ingredients I need to prepare tonight's dinner at a good price. Tell me about coupons I can use at the supermarket I'm planning to go to and the best payment method," and provides appropriate information immediately.

[0129] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0130] Step 1:

[0131] The user inputs information using natural language via a smartphone or smart glasses. This input is sent to the server. A natural language processing engine analyzes the input text to identify the user's intent and desires. This process utilizes natural language processing to break down the text and extract keywords and requirements. For example, if a request includes "cheapest detergent," the keywords "cheapest" and "detergent" are identified.

[0132] Step 2:

[0133] The server collects relevant data from the internet based on the identified keywords. Here, web scraping techniques are used to obtain price and discount information from various websites. This method efficiently collects the latest data that matches the user's requirements. The collected data is stored in a database and used for analysis.

[0134] Step 3:

[0135] The server analyzes the collected data and generates the most suitable suggestions for the user. Using a generative AI model, it builds personalized suggestions based on the collected information, including price comparisons of products and the user's purchase history. This analysis generates suggestions for the most suitable products, payment options, and discount information for the user.

[0136] Step 4:

[0137] The generated suggestions are sent from the server to the user's terminal. The terminal presents this information visually or audibly through its user interface. This allows the user to easily review the suggestions. The terminal formats the received data and displays it in a way that is easy for the user to understand.

[0138] Step 5:

[0139] Users can retrieve the presented information and provide feedback on it. This feedback is sent to the server via the device. The server uses the collected feedback to improve the accuracy of suggestions and personalization through algorithmic processing. Machine learning is performed based on the feedback to improve the quality of information provided in the future.

[0140] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0141] This invention is an information provision system that takes into account the user's emotional state. The system aims to provide more personalized suggestions by analyzing the user's natural language input and simultaneously evaluating the user's intentions and emotions.

[0142] 1. Processing user input and emotion recognition

[0143] Users input requests and objectives into the device using natural language, and their emotional state is collected based on their tone of voice, facial expressions, and input content. The device then sends this data to a server.

[0144] 2. Utilization of Natural Language Processing and Sentiment Engines

[0145] The server analyzes the user's input data using natural language processing techniques to identify the user's intent and recognizes the user's emotional state using an emotion engine. The emotional information obtained in this step is used to customize the suggested content.

[0146] 3. Collection and analysis of relevant data

[0147] The server collects necessary data from the internet based on the user's intent and analyzes it using AI technology. This data collection includes product information, price comparisons, and service evaluations.

[0148] 4. Personalizing suggestions

[0149] The server comprehensively considers the acquired data and the user's emotional state to generate the most suitable suggestions for the user. These suggestions include a message tone and content that reflects the user's emotions.

[0150] 5. Information presentation and interface optimization

[0151] The terminal displays suggestions received from the server on the user interface. During this process, the interface's color scheme and message tone are adjusted based on the user's emotions. For example, if the user is feeling stressed, calm colors and gentle language will be used.

[0152] 6. Gathering feedback and further learning

[0153] When a user accepts a suggestion and inputs the result as feedback on their device, the system learns further. Based on the feedback, the server updates the model to improve sentiment recognition and suggestion accuracy.

[0154] For example, if a user enters "I'm looking for relaxation items to relieve stress on my days off," the system will sense fatigue from their voice. Based on this, it will generate suggestions, including products with high relaxation effects and special offers for comfortable delivery services, which will be presented to the user in a gentle tone through their device.

[0155] This system allows users to receive suggestions that best suit their emotional state, resulting in a more satisfying experience.

[0156] The following describes the processing flow.

[0157] Step 1:

[0158] Users input the information and purpose they are looking for into the device using natural language. Simultaneously, the user's voice tone and facial expressions are collected as emotional data.

[0159] Step 2:

[0160] The device sends the user's natural language input and associated sentiment data to the server. This data forms the basis for the system to understand the user's needs and emotional state.

[0161] Step 3:

[0162] The server analyzes the received data using natural language processing techniques to identify the user's intent. Simultaneously, it uses an emotion engine to analyze the user's emotional state.

[0163] Step 4:

[0164] The server collects relevant data from the internet based on the identified user's intentions and emotional state. This data can range from product information and service details to ratings and price comparisons.

[0165] Step 5:

[0166] The server uses AI technology to analyze the collected data and generate suggestions that are best suited to the user's intentions and emotional state. This process is highly sensitive to changes in emotions and dynamically adjusts the suggestions accordingly.

[0167] Step 6:

[0168] The server sends the generated suggestions to the terminal. The suggestions include a message tone that suits the user's emotions and highlights the benefits of specific products.

[0169] Step 7:

[0170] The device presents suggestions to the user through its interface. The screen's color scheme and message tone may change depending on the user's mood.

[0171] Step 8:

[0172] The user reviews the presented suggestions and selects an action. Based on this selection, the user can input feedback on the suggestions into the device.

[0173] Step 9:

[0174] The device sends user feedback to the server. This allows the entire system to learn and contributes to improving the accuracy of future suggestions.

[0175] Step 10:

[0176] The server updates its sentiment recognition model and suggestion generation algorithm based on the collected feedback. This allows for more precise and user-friendly suggestions.

[0177] (Example 2)

[0178] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0179] Modern information delivery systems generally only provide one-way information in response to user requests, lacking personalization that takes into account the user's emotional state. As a result, the user experience is unsatisfactory, and it is difficult to meet the diverse needs of users. This invention aims to solve these problems and provide more comprehensive information that simultaneously considers the user's intentions and emotional state.

[0180] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0181] In this invention, the server includes means for analyzing user input using natural language processing technology to identify the user's intent, means for recognizing the user's emotional state based on voice tone and facial expression data acquired during user input, and means for automatically collecting and analyzing relevant information from the internet. This makes it possible to provide the user with optimal information that takes into account the user's intent and emotional state, thereby providing a more personalized and satisfying experience.

[0182] "User input" refers to requests, objectives, and information that a user transmits to the system via a terminal.

[0183] "Natural language processing technology" is a technology that enables computers to analyze and understand human language.

[0184] "Identifying intent" refers to accurately understanding the user's requests and objectives based on their input.

[0185] "Voice tone" refers to the pitch, volume, and overall tone of voice used to express emotions when a user speaks.

[0186] "Facial expression data" refers to information that records changes in a user's face, indicating their emotions and reactions.

[0187] "Emotional state" refers to the psychological state or emotional characteristics that a user exhibits when entering information.

[0188] "Related information" refers to data and information related to the user's requests and objectives, and is obtained from the internet or databases.

[0189] "Collecting from the internet" means obtaining necessary data from online sources.

[0190] "Analyzing" is the act of examining collected data to find the desired meaning and relationships.

[0191] "Optimal information" refers to information that provides the most suitable suggestions and answers to the user, based on the user's intentions and emotional state.

[0192] "Feedback" refers to a user's reaction to or evaluation of suggestions or information provided.

[0193] "Learning" is the process by which a system improves itself based on collected feedback and other factors.

[0194] A "user terminal" is a computer graphics or mobile device used by a user to input information or receive results.

[0195] This system incorporates technology designed to provide information that takes into account the user's emotional state. It begins with the user entering a request in natural language into a terminal. In addition to the input, the terminal collects supplementary information such as voice tone and facial expression data. This collected data is then transmitted to a server via secure communication (e.g., TLS protocol).

[0196] The server uses natural language processing technology to analyze user input and identify the user's intent. A generative AI model is used, and the prompt "Analyze the user's request and identify their intent" is input. Simultaneously, the server uses an emotion engine to recognize the emotional state from voice tone and facial expression data.

[0197] Relevant information is automatically collected via the internet and internal databases, and the necessary data is analyzed using AI technology. This process utilizes product information APIs, among others. Based on the collected information and recognized emotional states, the server generates information best suited to the user and creates personalized suggestions.

[0198] The suggestions are then sent to the user's terminal and displayed on the user interface with appropriate colors and tones. This allows the user to receive information optimized according to their emotional state. For example, if a user enters "I'm looking for relaxation items to relieve stress on my days off," the system will suggest relaxation products based on the user's emotions and display the information in a calming tone.

[0199] Finally, the user inputs feedback on the suggested information into the device. The server then uses this feedback to collect data and update the AI ​​model to improve the accuracy of suggestions and sentiment recognition. By repeating this process, the system is continuously improved, enhancing the quality of the user experience.

[0200] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0201] Step 1:

[0202] The user inputs information into the device using natural language. The device records the user's requests and objectives as text data, while simultaneously collecting voice tone and facial expression data using the microphone and camera. All of this input data is organized and prepared as a single data package for the next processing step. At this stage, the input content is output as text data, and accompanying information is output as voice and facial expression data.

[0203] Step 2:

[0204] The terminal sends the organized data package to the server. Data security is ensured using protocols such as TLS. The server receives the transmitted data and prepares for the next analysis step. The input is the data package from the terminal, and the output is the data converted into an analyzable format.

[0205] Step 3:

[0206] The server uses natural language processing techniques to analyze text data and identify the user's intent. Here, the generative AI model is input with the prompt, "Analyze the user's request and identify their intent." The server then outputs keywords and objective information related to the user's intent as part of its analysis.

[0207] Step 4:

[0208] The server uses an emotion engine to recognize the user's emotional state from voice tone and facial expression data. Voice analysis algorithms and facial recognition technology are used. Input is voice and facial expression data, and output is tags and numerical indicators that show emotional tendencies.

[0209] Step 5:

[0210] The server collects relevant information from the internet and databases based on the user's intent and emotional state. This process involves the execution of product information APIs and online search algorithms. Input consists of keywords and intent information, while output is a list of relevant information.

[0211] Step 6:

[0212] The server generates information best suited to the user based on collected data and recognized emotional states. An AI model analyzes the data and determines the most appropriate suggestions. The input is collected information and emotional data, and the output is personalized suggestions.

[0213] Step 7:

[0214] The device displays the received suggestions on the user interface. The tone and color scheme of the suggestions are adjusted according to the user's emotional state. The input is the suggested content, and the output is the visual presentation of the information.

[0215] Step 8:

[0216] The user provides feedback on the presented information. The device records this feedback and sends it to the server. This data is used to improve the accuracy of future suggestions. The input is the user's comments and evaluations, and the output is the recorded feedback data.

[0217] Step 9:

[0218] The server analyzes the feedback and updates the AI ​​model. This process continuously improves sentiment recognition and suggestion accuracy. The input is feedback data, and the output is the updated AI model.

[0219] (Application Example 2)

[0220] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0221] Conventional information delivery systems have difficulty providing personalized suggestions that take into account the user's emotional state, resulting in challenges in improving user satisfaction. Furthermore, because the suggested content is not optimized for the user's emotional state at the time, it has been difficult to improve the user experience.

[0222] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0223] In this invention, the server includes means for analyzing user input using natural language processing to simultaneously identify the user's intent and emotional state, means for acquiring audio and video data to recognize the user's emotional state, and means for automatically collecting relevant information from the internet and analyzing it using artificial intelligence technology. This makes it possible to provide personalized suggestions that are optimal for the user's emotional state.

[0224] "User input" refers to the expression of requests or objectives made by a user through a device, either via voice or text.

[0225] "Natural language processing" is the technology that enables computers to understand, analyze, and process human language.

[0226] "Intention" refers to the purpose or wish that the user is trying to convey through their input.

[0227] "Emotional state" refers to information that indicates the emotions a user is feeling at a particular point in time.

[0228] "Audio data" refers to audio information that records the user's voice.

[0229] "Video data" refers to video information that records the user's facial expressions and movements.

[0230] "Automatically collecting relevant information from the internet" refers to the process of automatically retrieving information that is publicly available online.

[0231] "Artificial intelligence technology" is a technology that allows computers to mimic some aspects of human intelligence.

[0232] A "user interface" refers to the screens and methods of operation that a user uses when interacting with a system.

[0233] "Feedback" refers to the opinions and responses that users provide in response to suggestions they receive.

[0234] "Proposal accuracy" is an indicator of how well the proposals are tailored to the user's needs and feelings.

[0235] The system for realizing this invention uses a user terminal such as a smartphone or tablet and a server in the cloud. When the user inputs voice or text into the terminal, the terminal uses its built-in microphone and camera to collect data on the user's voice and facial expressions. This information is transmitted to the server via the internet.

[0236] On the server, natural language processing software (e.g., Google Cloud Natural Language API) is first used to analyze the user's input text and identify the user's intent. Simultaneously, an emotion recognition engine recognizes emotional states from audio and video data. This analysis utilizes speech recognition APIs (e.g., Google Cloud Speech-to-Text API) and facial expression recognition libraries (e.g., OpenCV).

[0237] Based on the user's intent and emotional state derived from this data, the server collects relevant information from the internet and analyzes it using artificial intelligence technology. The analysis utilizes machine learning models to generate personalized suggestions best suited to the user. These suggestions are sent to the user's device, and the content displayed in the user interface is adjusted in terms of color scheme and message tone to match the user's emotional state.

[0238] For example, if a user intends to relax on their day off and the server detects stress from their tone of voice, it will suggest products with relaxing effects and comfortable delivery options in a gentle tone.

[0239] An example of an input prompt for a generative AI model could be, "Please create a personalized suggestion that considers products and benefits to a user who wants to relax." Based on this prompt, the AI ​​can generate optimal suggestions that match the user's emotions.

[0240] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0241] Step 1:

[0242] Users input information into their smartphones or tablets via voice or text. The device receives this input and simultaneously captures audio and video data using its built-in microphone and camera. This collects data about the user's intent and emotional data based on voice tone and facial expressions.

[0243] Step 2:

[0244] The device converts the collected audio data into text using a speech recognition API and sends it to the server. At the same time, video data is also sent for emotion recognition and used as material to evaluate the user's emotional state from their facial expressions. This forms the input dataset.

[0245] Step 3:

[0246] The server analyzes the transcribed user input using natural language processing techniques to identify the user's specific intentions. It also uses an emotion engine to recognize emotional states based on audio and video data. The input data consists of speech recognition results and video data, while the output provides metadata of the user's intentions and emotions.

[0247] Step 4:

[0248] The server collects relevant information from the internet based on the analyzed intent and sentiment metadata. It uses AI technology to analyze product information and reviews and generate personalized suggestions for the user. In this process, the input relevant information is processed into personalized suggestions as output.

[0249] Step 5:

[0250] The generated suggestions are sent to the device to be presented with a tone and color scheme adjusted based on the user's emotional state. The device then appropriately displays the received suggestions on the user interface. At this time, the displayed content will have its color scheme and tone adjusted based on the input emotional metadata.

[0251] Step 6:

[0252] The user sends feedback on the presented suggestions from their device to the server. The server collects this feedback data as training material and uses it to improve the accuracy of the suggestions and the sentiment recognition model. This feedback loop improves the performance of the output system in subsequent uses.

[0253] For the generating AI model, the prompt "Please create a personalized suggestion that considers products and benefits to recommend when the user is feeling relaxed" is used to generate AI-based suggestions.

[0254] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0255] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0256] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0257] [Second Embodiment]

[0258] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0259] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0260] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0261] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0262] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0263] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0264] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0265] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0266] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0267] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0268] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0269] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0270] This invention is a system that supports users in efficiently acquiring information and making optimal decisions in their daily lives. The system operates with its components working together as follows:

[0271] 1. Processing user input

[0272] Users input their wishes and objectives in natural language via their devices. For example, "I want to have a party this weekend, so I'd like information on ingredients and venues."

[0273] 2. Analysis using natural language processing

[0274] The server receives user input sent from the terminal and analyzes it using natural language processing technology. This analysis identifies the user's intent and the information they need.

[0275] 3. Collection of related data

[0276] The server collects necessary data from the internet based on the analysis results. Specifically, it acquires website data to obtain information about products and services that users are looking for, as well as the best prices.

[0277] 4. Proposal generation

[0278] The server analyzes the collected data and generates suggestions that best meet the user's needs. This includes price comparisons, location guidance, and coupon information.

[0279] 5. Presenting information to the user

[0280] The terminal displays suggested information received from the server to the user. The user can also ask additional questions using the voice assistant.

[0281] 6. Gathering and Learning from Feedback

[0282] The user acts based on the presented suggestion. The device records the result and sends it to the server. The server learns from this feedback and uses it as data to improve future suggestions.

[0283] As a specific example, when a user plans to purchase daily necessities, the system is utilized as follows. When the user inputs "I want to buy detergent and shampoo by the end of this week", the server analyzes the request, compares the information of nearby stores and online prices, and proposes the optimal purchase option. Furthermore, the server can refer to the user's past purchase history and recommend favorite brands. The terminal presents this information to the user, and the user makes a purchase based on the proposal. The user's purchase behavior is sent to the server as feedback and utilized to improve the accuracy of future proposals.

[0284] With such a configuration, the user can efficiently obtain information without much effort and make an optimal choice.

[0285] The following describes the processing flow.

[0286] Step 1:

[0287] The user inputs wishes and requirements related to daily life in natural language to the terminal. This input becomes the initial trigger for this system.

[0288] Step 2:

[0289] The terminal sends the user's input to the server as digital data. This data is necessary for analyzing the user's intention.

[0290] Step 3:

[0291] The server applies natural language processing technology to analyze the user's input. Through this analysis, the user's requirements and purposes are specifically identified, and the types of necessary information are determined.

[0292] Step 4:

[0293] The server collects relevant data from the Internet based on the analysis results. This collection work is carried out through API access or web scraping.

[0294] Step 5:

[0295] The server uses AI technology to analyze the collected data and generate optimal recommendations for the user. This analysis includes price comparisons, quality evaluations, and user reviews.

[0296] Step 6:

[0297] The server sends the generated suggestion to the terminal. This suggestion is a structured version of the information the user is looking for.

[0298] Step 7:

[0299] The terminal displays suggestions from the server to the user via a user interface. The user can then ask more detailed questions by activating the voice assistant.

[0300] Step 8:

[0301] The user makes decisions based on the suggestions presented on the device. For example, they might choose to make a purchase at a suggested store.

[0302] Step 9:

[0303] After a user's action, the device records the result and sends the user's feedback to the server.

[0304] Step 10:

[0305] The server uses feedback to update the AI ​​model and learn to improve the accuracy of future suggestions.

[0306] (Example 1)

[0307] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0308] In modern society, the means for efficiently collecting the information required by users and supporting optimal decision-making are limited. In particular, in situations where quick and appropriate information acquisition is required in daily activities, it is difficult for users to identify the optimal option from many choices. Also, in a situation of excessive information, a mechanism for providing personalized proposals based on individual needs and preferences has not been sufficiently established.

[0309] The specific processing by the specific processing unit 290 of the data processing apparatus 12 in Example 1 is realized by the following means.

[0310] In this invention, the server includes means for analyzing a natural language input received from a user via an information device and specifying the user's intention, means for automatically collecting and analyzing related information from a database on the Internet by an information processing apparatus, and means for generating a proposal suitable for the user's request based on the collected information. Thereby, it becomes possible to obtain optimal information and provide proposals based on the user's intention.

[0311] "Information device" is a general term for terminals and devices used by users to input information.

[0312] "Natural language input" is a method by which a user provides information to an information device using natural words and sentences.

[0313] "Information processing apparatus" is a computing device used for collecting, analyzing, and processing digital data.

[0314] "Database" is a structured information aggregate for storing necessary information organizationally and enabling access.

[0315] "Related information" is data and facts related to the user's intention and request.

[0316] "Proposal" means presenting users with convenient and optimal options or actions based on the information collected.

[0317] "Feedback information" refers to information about the results and reactions related to user actions and choices.

[0318] "Personalized suggestions" refer to suggestions that are specially customized to match the user's past behavior history and preferences.

[0319] This invention is a system that supports the efficient acquisition of information and the optimization of decision-making in the user's daily life. This system is implemented using information equipment, servers, and related software technologies.

[0320] First, the user inputs their purpose or request in natural language via an information device. For example, they might input a request such as, "I'm looking for a place to meet up with friends this weekend, and I'd like to know the best restaurant." This input is received by the terminal, and the user's response is sent to the server as a prompt. An example of such a prompt would be, "Please recommend a restaurant for the weekend."

[0321] Next, the server utilizes a generative AI model and natural language processing techniques to analyze user input. Through this analysis, the server identifies the user's intent and, based on that information, collects relevant data from the internet. This data collection can be done using database queries or web scraping techniques. Specifically, AI technologies known as generative AI models and web scraping tools are used.

[0322] The server then processes the collected information using advanced data analysis techniques to generate optimal suggestions for the user. For example, based on the collected restaurant information, it provides personalized suggestions based on the user's preferences and past reviews. These suggestions include price comparisons and restaurant location information.

[0323] Finally, the terminal displays the suggestions received from the server to the user. The user makes a decision based on this information and sends the result and feedback back to the server via the terminal. The server analyzes this feedback information and uses it to improve the accuracy of the suggestions. This adjusts future suggestions to be more accurate and user-oriented.

[0324] This embodiment allows users to intuitively and efficiently obtain the necessary information and make decisions without having to perform cumbersome information searches.

[0325] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0326] Step 1:

[0327] The user uses an information device to input specific purposes or wishes in natural language. The input text is received on the user's device, and a prompt message such as "Please recommend some restaurants for the weekend" is generated. The input natural language text becomes the basic data for subsequent processing.

[0328] Step 2:

[0329] The terminal sends user input to the server. During this process, the input data is securely encrypted using the SSL / TLS protocol and transmitted. The transmitted prompt message is awaiting analysis by the server.

[0330] Step 3:

[0331] The server analyzes the received natural language input using a generative AI model. Specifically, it uses natural language processing techniques to identify the intent of the input sentence and extract keywords. The input is a prompt sentence, and the output is a set of analyzed keywords and themes (e.g., "restaurant," "weekend," etc.).

[0332] Step 4:

[0333] The server collects relevant data from the internet based on the analyzed keywords. Here, web scraping tools are used to gather specific store information, price information, and other data from publicly available information and databases on the internet. The input is keywords, and the output is a collection of relevant data.

[0334] Step 5:

[0335] The server processes the collected data using data analysis techniques to generate personalized recommendations for the user. For example, it compares price information from a database to suggest the most cost-effective restaurant. It also includes personalized advice based on past user preferences. The input is the collected raw data, and the output is specific recommendation information.

[0336] Step 6:

[0337] The proposed information is sent from the server to the terminal. This transmission involves formatting the data (e.g., JSON format) to ensure it is suitable for display. The input is the proposed information, and the output is the data sent to the user's terminal.

[0338] Step 7:

[0339] The device presents the user with appropriately formatted suggestion information. The user can view this information and, if necessary, request further details using the voice assistant or touch interface. The input is formatted data, and the output is visual or audible feedback to the user.

[0340] Step 8:

[0341] Users make decisions based on the information presented and provide feedback on the results and their impressions through their device. This feedback is used to improve future proposals. The input is the user's experience and choices, and the output is feedback data.

[0342] Step 9:

[0343] The server receives feedback data and uses machine learning techniques to improve the system's proposed algorithm. This process enables more accurate personalized recommendations for subsequent uses. The input is feedback data, and the output is the improved algorithm model.

[0344] (Application Example 1)

[0345] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0346] When users choose products and services in their daily lives, it is difficult for them to quickly and accurately find the best option from a vast amount of information. Furthermore, effectively utilizing payment options and discount information is also challenging. This complicates the user's purchasing experience and hinders optimal decision-making.

[0347] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0348] In this invention, the server includes means for analyzing user input using natural language processing to identify the user's intent, means for generating optimal suggestions for the user using the collected data, and means for providing the user with optimal payment options and discount information in real time. This enables users to efficiently obtain relevant information and quickly make optimal choices in their daily purchasing activities.

[0349] "User input" refers to the act of a user sending information to a system via a terminal, and the content of that information, including natural language and instructions.

[0350] "Natural language processing" is a technology that uses computers to analyze human language and understand user intent and information.

[0351] "Automatically collecting relevant data from the internet" refers to the process of automatically searching for and retrieving relevant information that exists on a network.

[0352] "Generating suggestions" is the act of organizing and presenting the most suitable options and information for the user based on collected data.

[0353] "User terminal" refers to a digital device used by a user to receive information, and includes smartphones, computers, and other similar devices.

[0354] "Improving the accuracy of suggestions based on feedback" is a process of analyzing user responses and results to improve the quality of information provided in the future.

[0355] "Payment options" refer to the payment methods and conditions available when purchasing goods or services.

[0356] "Discount information" refers to information about price reductions or benefits that apply under specific conditions.

[0357] The system for realizing this invention consists of a user-owned digital device (such as a smartphone or smart glasses) and a server connected via the internet. The server receives user input, performs natural language processing, and analyzes the user's intentions and requests. This analysis uses natural language processing libraries (e.g., spaCy, Google NLP API).

[0358] The server automatically collects information on relevant products and services from the internet based on the analysis results. Web scraping tools (e.g., BeautifulSoup, Selenium) are used for this information collection. The collected data is then used to generate suggestions best suited to the user's needs. Generative AI models may be used to personalize these suggestions.

[0359] The user's device displays suggested information received from the server via voice and visuals. The user reviews this information and provides additional instructions as needed, such as through voice input. Furthermore, payment options and discount information related to the products and services the user is considering are also provided in real time.

[0360] As a concrete example, consider a scenario where a user is planning their grocery shopping trip using smart glasses. If the user voice-inputs, "I want to know what the cheapest detergent is right now," the system analyzes the information and immediately displays the latest price information and available discounts.

[0361] By utilizing generative AI models in this process, it becomes possible to provide optimal suggestions to users quickly and with high accuracy, improving the user's purchasing experience. It also accepts prompt messages such as, "I want to buy the ingredients I need to prepare tonight's dinner at a good price. Tell me about coupons I can use at the supermarket I'm planning to go to and the best payment method," and provides appropriate information immediately.

[0362] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0363] Step 1:

[0364] The user inputs information using natural language via a smartphone or smart glasses. This input is sent to the server. A natural language processing engine analyzes the input text to identify the user's intent and desires. This process utilizes natural language processing to break down the text and extract keywords and requirements. For example, if a request includes "cheapest detergent," the keywords "cheapest" and "detergent" are identified.

[0365] Step 2:

[0366] The server collects relevant data from the internet based on the identified keywords. Here, web scraping techniques are used to obtain price and discount information from various websites. This method efficiently collects the latest data that matches the user's requirements. The collected data is stored in a database and used for analysis.

[0367] Step 3:

[0368] The server analyzes the collected data and generates the most suitable suggestions for the user. Using a generative AI model, it builds personalized suggestions based on the collected information, including price comparisons of products and the user's purchase history. This analysis generates suggestions for the most suitable products, payment options, and discount information for the user.

[0369] Step 4:

[0370] The generated suggestions are sent from the server to the user's terminal. The terminal presents this information visually or audibly through its user interface. This allows the user to easily review the suggestions. The terminal formats the received data and displays it in a way that is easy for the user to understand.

[0371] Step 5:

[0372] Users can retrieve the presented information and provide feedback on it. This feedback is sent to the server via the device. The server uses the collected feedback to improve the accuracy of suggestions and personalization through algorithmic processing. Machine learning is performed based on the feedback to improve the quality of information provided in the future.

[0373] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0374] This invention is an information provision system that takes into account the user's emotional state. The system aims to provide more personalized suggestions by analyzing the user's natural language input and simultaneously evaluating the user's intentions and emotions.

[0375] 1. Processing user input and emotion recognition

[0376] Users input requests and objectives into the device using natural language, and their emotional state is collected based on their tone of voice, facial expressions, and input content. The device then sends this data to a server.

[0377] 2. Utilization of Natural Language Processing and Sentiment Engines

[0378] The server analyzes the user's input data using natural language processing techniques to identify the user's intent and recognizes the user's emotional state using an emotion engine. The emotional information obtained in this step is used to customize the suggested content.

[0379] 3. Collection and analysis of relevant data

[0380] The server collects necessary data from the internet based on the user's intent and analyzes it using AI technology. This data collection includes product information, price comparisons, and service evaluations.

[0381] 4. Personalizing suggestions

[0382] The server comprehensively considers the acquired data and the user's emotional state to generate the most suitable suggestions for the user. These suggestions include a message tone and content that reflects the user's emotions.

[0383] 5. Information presentation and interface optimization

[0384] The terminal displays suggestions received from the server on the user interface. During this process, the interface's color scheme and message tone are adjusted based on the user's emotions. For example, if the user is feeling stressed, calm colors and gentle language will be used.

[0385] 6. Gathering feedback and further learning

[0386] When a user accepts a suggestion and inputs the result as feedback on their device, the system learns further. Based on the feedback, the server updates the model to improve sentiment recognition and suggestion accuracy.

[0387] For example, if a user enters "I'm looking for relaxation items to relieve stress on my days off," the system will sense fatigue from their voice. Based on this, it will generate suggestions, including products with high relaxation effects and special offers for comfortable delivery services, which will be presented to the user in a gentle tone through their device.

[0388] This system allows users to receive suggestions that best suit their emotional state, resulting in a more satisfying experience.

[0389] The following describes the processing flow.

[0390] Step 1:

[0391] Users input the information and purpose they are looking for into the device using natural language. Simultaneously, the user's voice tone and facial expressions are collected as emotional data.

[0392] Step 2:

[0393] The device sends the user's natural language input and associated sentiment data to the server. This data forms the basis for the system to understand the user's needs and emotional state.

[0394] Step 3:

[0395] The server analyzes the received data using natural language processing techniques to identify the user's intent. Simultaneously, it uses an emotion engine to analyze the user's emotional state.

[0396] Step 4:

[0397] The server collects relevant data from the internet based on the identified user's intentions and emotional state. This data can range from product information and service details to ratings and price comparisons.

[0398] Step 5:

[0399] The server uses AI technology to analyze the collected data and generate suggestions that are best suited to the user's intentions and emotional state. This process is highly sensitive to changes in emotions and dynamically adjusts the suggestions accordingly.

[0400] Step 6:

[0401] The server sends the generated suggestions to the terminal. The suggestions include a message tone that suits the user's emotions and highlights the benefits of specific products.

[0402] Step 7:

[0403] The device presents suggestions to the user through its interface. The screen's color scheme and message tone may change depending on the user's mood.

[0404] Step 8:

[0405] The user reviews the presented suggestions and selects an action. Based on this selection, the user can input feedback on the suggestions into the device.

[0406] Step 9:

[0407] The device sends user feedback to the server. This allows the entire system to learn and contributes to improving the accuracy of future suggestions.

[0408] Step 10:

[0409] The server updates its sentiment recognition model and suggestion generation algorithm based on the collected feedback. This allows for more precise and user-friendly suggestions.

[0410] (Example 2)

[0411] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0412] Modern information delivery systems generally only provide one-way information in response to user requests, lacking personalization that takes into account the user's emotional state. As a result, the user experience is unsatisfactory, and it is difficult to meet the diverse needs of users. This invention aims to solve these problems and provide more comprehensive information that simultaneously considers the user's intentions and emotional state.

[0413] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0414] In this invention, the server includes means for analyzing user input using natural language processing technology to identify the user's intent, means for recognizing the user's emotional state based on voice tone and facial expression data acquired during user input, and means for automatically collecting and analyzing relevant information from the internet. This makes it possible to provide the user with optimal information that takes into account the user's intent and emotional state, thereby providing a more personalized and satisfying experience.

[0415] "User input" refers to requests, objectives, and information that a user transmits to the system via a terminal.

[0416] "Natural language processing technology" is a technology that enables computers to analyze and understand human language.

[0417] "Identifying intent" refers to accurately understanding the user's requests and objectives based on their input.

[0418] "Voice tone" refers to the pitch, volume, and overall tone of voice used to express emotions when a user speaks.

[0419] "Facial expression data" refers to information that records changes in a user's face, indicating their emotions and reactions.

[0420] "Emotional state" refers to the psychological state or emotional characteristics that a user exhibits when entering information.

[0421] "Related information" refers to data and information related to the user's requests and objectives, and is obtained from the internet or databases.

[0422] "Collecting from the internet" means obtaining necessary data from online sources.

[0423] "Analyzing" is the act of examining collected data to find the desired meaning and relationships.

[0424] "Optimal information" refers to information that provides the most suitable suggestions and answers to the user, based on the user's intentions and emotional state.

[0425] "Feedback" refers to a user's reaction to or evaluation of suggestions or information provided.

[0426] "Learning" is the process by which a system improves itself based on collected feedback and other factors.

[0427] A "user terminal" is a computer graphics or mobile device used by a user to input information or receive results.

[0428] This system incorporates technology designed to provide information that takes into account the user's emotional state. It begins with the user entering a request in natural language into a terminal. In addition to the input, the terminal collects supplementary information such as voice tone and facial expression data. This collected data is then transmitted to a server via secure communication (e.g., TLS protocol).

[0429] The server uses natural language processing technology to analyze user input and identify the user's intent. A generative AI model is used, and the prompt "Analyze the user's request and identify their intent" is input. Simultaneously, the server uses an emotion engine to recognize the emotional state from voice tone and facial expression data.

[0430] Relevant information is automatically collected via the internet and internal databases, and the necessary data is analyzed using AI technology. This process utilizes product information APIs, among others. Based on the collected information and recognized emotional states, the server generates information best suited to the user and creates personalized suggestions.

[0431] The suggestions are then sent to the user's terminal and displayed on the user interface with appropriate colors and tones. This allows the user to receive information optimized according to their emotional state. For example, if a user enters "I'm looking for relaxation items to relieve stress on my days off," the system will suggest relaxation products based on the user's emotions and display the information in a calming tone.

[0432] Finally, the user inputs feedback on the suggested information into the device. The server then uses this feedback to collect data and update the AI ​​model to improve the accuracy of suggestions and sentiment recognition. By repeating this process, the system is continuously improved, enhancing the quality of the user experience.

[0433] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0434] Step 1:

[0435] The user inputs information into the device using natural language. The device records the user's requests and objectives as text data, while simultaneously collecting voice tone and facial expression data using the microphone and camera. All of this input data is organized and prepared as a single data package for the next processing step. At this stage, the input content is output as text data, and accompanying information is output as voice and facial expression data.

[0436] Step 2:

[0437] The terminal sends the organized data package to the server. Data security is ensured using protocols such as TLS. The server receives the transmitted data and prepares for the next analysis step. The input is the data package from the terminal, and the output is the data converted into an analyzable format.

[0438] Step 3:

[0439] The server uses natural language processing techniques to analyze text data and identify the user's intent. Here, the generative AI model is input with the prompt, "Analyze the user's request and identify their intent." The server then outputs keywords and objective information related to the user's intent as part of its analysis.

[0440] Step 4:

[0441] The server uses an emotion engine to recognize the user's emotional state from voice tone and facial expression data. Voice analysis algorithms and facial recognition technology are used. Input is voice and facial expression data, and output is tags and numerical indicators that show emotional tendencies.

[0442] Step 5:

[0443] The server collects relevant information from the internet and databases based on the user's intent and emotional state. This process involves the execution of product information APIs and online search algorithms. Input consists of keywords and intent information, while output is a list of relevant information.

[0444] Step 6:

[0445] The server generates information best suited to the user based on collected data and recognized emotional states. An AI model analyzes the data and determines the most appropriate suggestions. The input is collected information and emotional data, and the output is personalized suggestions.

[0446] Step 7:

[0447] The device displays the received suggestions on the user interface. The tone and color scheme of the suggestions are adjusted according to the user's emotional state. The input is the suggested content, and the output is the visual presentation of the information.

[0448] Step 8:

[0449] The user provides feedback on the presented information. The device records this feedback and sends it to the server. This data is used to improve the accuracy of future suggestions. The input is the user's comments and evaluations, and the output is the recorded feedback data.

[0450] Step 9:

[0451] The server analyzes the feedback and updates the AI ​​model. This process continuously improves sentiment recognition and suggestion accuracy. The input is feedback data, and the output is the updated AI model.

[0452] (Application Example 2)

[0453] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0454] Conventional information delivery systems have difficulty providing personalized suggestions that take into account the user's emotional state, resulting in challenges in improving user satisfaction. Furthermore, because the suggested content is not optimized for the user's emotional state at the time, it has been difficult to improve the user experience.

[0455] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0456] In this invention, the server includes means for analyzing user input using natural language processing to simultaneously identify the user's intent and emotional state, means for acquiring audio and video data to recognize the user's emotional state, and means for automatically collecting relevant information from the internet and analyzing it using artificial intelligence technology. This makes it possible to provide personalized suggestions that are optimal for the user's emotional state.

[0457] "User input" refers to the expression of requests or objectives made by a user through a device, either via voice or text.

[0458] "Natural language processing" is the technology that enables computers to understand, analyze, and process human language.

[0459] "Intention" refers to the purpose or wish that the user is trying to convey through their input.

[0460] "Emotional state" refers to information that indicates the emotions a user is feeling at a particular point in time.

[0461] "Audio data" refers to audio information that records the user's voice.

[0462] "Video data" refers to video information that records the user's facial expressions and movements.

[0463] "Automatically collecting relevant information from the internet" refers to the process of automatically retrieving information that is publicly available online.

[0464] "Artificial intelligence technology" is a technology that allows computers to mimic some aspects of human intelligence.

[0465] A "user interface" refers to the screens and methods of operation that a user uses when interacting with a system.

[0466] "Feedback" refers to the opinions and responses that users provide in response to suggestions they receive.

[0467] "Proposal accuracy" is an indicator of how well the proposals are tailored to the user's needs and feelings.

[0468] The system for realizing this invention uses a user terminal such as a smartphone or tablet and a server in the cloud. When the user inputs voice or text into the terminal, the terminal uses its built-in microphone and camera to collect data on the user's voice and facial expressions. This information is transmitted to the server via the internet.

[0469] On the server, natural language processing software (e.g., Google Cloud Natural Language API) is first used to analyze the user's input text and identify the user's intent. Simultaneously, an emotion recognition engine recognizes emotional states from audio and video data. This analysis utilizes speech recognition APIs (e.g., Google Cloud Speech-to-Text API) and facial expression recognition libraries (e.g., OpenCV).

[0470] Based on the user's intent and emotional state derived from this data, the server collects relevant information from the internet and analyzes it using artificial intelligence technology. The analysis utilizes machine learning models to generate personalized suggestions best suited to the user. These suggestions are sent to the user's device, and the content displayed in the user interface is adjusted in terms of color scheme and message tone to match the user's emotional state.

[0471] For example, if a user intends to relax on their day off and the server detects stress from their tone of voice, it will suggest products with relaxing effects and comfortable delivery options in a gentle tone.

[0472] An example of an input prompt for a generative AI model could be, "Please create a personalized suggestion that considers products and benefits to a user who wants to relax." Based on this prompt, the AI ​​can generate optimal suggestions that match the user's emotions.

[0473] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0474] Step 1:

[0475] Users input information into their smartphones or tablets via voice or text. The device receives this input and simultaneously captures audio and video data using its built-in microphone and camera. This collects data about the user's intent and emotional data based on voice tone and facial expressions.

[0476] Step 2:

[0477] The device converts the collected audio data into text using a speech recognition API and sends it to the server. At the same time, video data is also sent for emotion recognition and used as material to evaluate the user's emotional state from their facial expressions. This forms the input dataset.

[0478] Step 3:

[0479] The server analyzes the transcribed user input using natural language processing techniques to identify the user's specific intentions. It also uses an emotion engine to recognize emotional states based on audio and video data. The input data consists of speech recognition results and video data, while the output provides metadata of the user's intentions and emotions.

[0480] Step 4:

[0481] The server collects relevant information from the internet based on the analyzed intent and sentiment metadata. It uses AI technology to analyze product information and reviews and generate personalized suggestions for the user. In this process, the input relevant information is processed into personalized suggestions as output.

[0482] Step 5:

[0483] The generated suggestions are sent to the device to be presented with a tone and color scheme adjusted based on the user's emotional state. The device then appropriately displays the received suggestions on the user interface. At this time, the displayed content will have its color scheme and tone adjusted based on the input emotional metadata.

[0484] Step 6:

[0485] The user sends feedback on the presented suggestions from their device to the server. The server collects this feedback data as training material and uses it to improve the accuracy of the suggestions and the sentiment recognition model. This feedback loop improves the performance of the output system in subsequent uses.

[0486] For the generating AI model, the prompt "Please create a personalized suggestion that considers products and benefits to recommend when the user is feeling relaxed" is used to generate AI-based suggestions.

[0487] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0488] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0489] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0490] [Third Embodiment]

[0491] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0492] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0493] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0494] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0495] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0496] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0497] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0498] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0499] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0500] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0501] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0502] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0503] This invention is a system that supports users in efficiently acquiring information and making optimal decisions in their daily lives. The system operates with its components working together as follows:

[0504] 1. Processing user input

[0505] Users input their wishes and objectives in natural language via their devices. For example, "I want to have a party this weekend, so I'd like information on ingredients and venues."

[0506] 2. Analysis using natural language processing

[0507] The server receives user input sent from the terminal and analyzes it using natural language processing technology. This analysis identifies the user's intent and the information they need.

[0508] 3. Collection of related data

[0509] The server collects necessary data from the internet based on the analysis results. Specifically, it acquires website data to obtain information about products and services that users are looking for, as well as the best prices.

[0510] 4. Proposal generation

[0511] The server analyzes the collected data and generates suggestions that best meet the user's needs. This includes price comparisons, location guidance, and coupon information.

[0512] 5. Presenting information to the user

[0513] The terminal displays suggested information received from the server to the user. The user can also ask additional questions using the voice assistant.

[0514] 6. Gathering and Learning from Feedback

[0515] The user acts based on the presented suggestion. The device records the result and sends it to the server. The server learns from this feedback and uses it as data to improve future suggestions.

[0516] As a concrete example, if a user is planning to purchase daily necessities, the system can be used as follows: When the user inputs "I want to buy detergent and shampoo by this weekend," the server analyzes the request, compares information on nearby stores and online prices, and suggests the best purchase options. Furthermore, the server can refer to the user's past purchase history and recommend their favorite brands. The terminal presents this information to the user, who then makes a purchase based on the suggestions. The user's purchase behavior is sent to the server as feedback and used to improve the accuracy of future suggestions.

[0517] This configuration allows users to obtain information efficiently and without hassle, enabling them to make optimal choices.

[0518] The following describes the processing flow.

[0519] Step 1:

[0520] The user inputs their wishes and requests related to their daily life into the terminal using natural language. This input serves as the initial trigger for this system.

[0521] Step 2:

[0522] The terminal transmits user input as digital data to the server. This data is necessary to analyze the user's intent.

[0523] Step 3:

[0524] The server uses natural language processing techniques to analyze user input. This analysis specifically identifies the user's requests and objectives, and determines the type of information needed.

[0525] Step 4:

[0526] The server collects relevant data from the internet based on the analysis results. This data collection is performed through API access and web scraping.

[0527] Step 5:

[0528] The server uses AI technology to analyze the collected data and generate optimal recommendations for the user. This analysis includes price comparisons, quality evaluations, and user reviews.

[0529] Step 6:

[0530] The server sends the generated suggestion to the terminal. This suggestion is a structured version of the information the user is looking for.

[0531] Step 7:

[0532] The terminal displays suggestions from the server to the user via a user interface. The user can then ask more detailed questions by activating the voice assistant.

[0533] Step 8:

[0534] The user makes decisions based on the suggestions presented on the device. For example, they might choose to make a purchase at a suggested store.

[0535] Step 9:

[0536] After a user's action, the device records the result and sends the user's feedback to the server.

[0537] Step 10:

[0538] The server uses feedback to update the AI ​​model and learn to improve the accuracy of future suggestions.

[0539] (Example 1)

[0540] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0541] In modern society, there are limited means to efficiently gather the information users need and support them in making optimal decisions. In particular, in situations where quick and appropriate information acquisition is required in daily activities, users find it difficult to discern the best option from many choices. Furthermore, in an information-overload environment, mechanisms for providing personalized suggestions based on individual needs and preferences are not yet fully established.

[0542] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0543] In this invention, the server includes means for analyzing natural language input received from a user via an information device to identify the user's intent, means for automatically collecting and analyzing relevant information from a database on the Internet using an information processing device, and means for generating suggestions suitable for the user's request based on the collected information. This makes it possible to obtain optimal information and provide suggestions based on the user's intent.

[0544] "Information equipment" is a general term for terminals and devices that users use to input information.

[0545] "Natural language input" is a method by which users provide information to information devices using natural language and sentences.

[0546] An "information processing device" is a computing device used to collect, analyze, and process digital data.

[0547] A "database" is a structured collection of information used to systematically store and access necessary information.

[0548] "Relevant information" refers to data and facts related to the user's intentions and requests.

[0549] "Proposal" means presenting users with convenient and optimal options or actions based on the information collected.

[0550] "Feedback information" refers to information about the results and reactions related to user actions and choices.

[0551] "Personalized suggestions" refer to suggestions that are specially customized to match the user's past behavior history and preferences.

[0552] This invention is a system that supports the efficient acquisition of information and the optimization of decision-making in the user's daily life. This system is implemented using information equipment, servers, and related software technologies.

[0553] First, the user inputs their purpose or request in natural language via an information device. For example, they might input a request such as, "I'm looking for a place to meet up with friends this weekend, and I'd like to know the best restaurant." This input is received by the terminal, and the user's response is sent to the server as a prompt. An example of such a prompt would be, "Please recommend a restaurant for the weekend."

[0554] Next, the server utilizes a generative AI model and natural language processing techniques to analyze user input. Through this analysis, the server identifies the user's intent and, based on that information, collects relevant data from the internet. This data collection can be done using database queries or web scraping techniques. Specifically, AI technologies known as generative AI models and web scraping tools are used.

[0555] The server then processes the collected information using advanced data analysis techniques to generate optimal suggestions for the user. For example, based on the collected restaurant information, it provides personalized suggestions based on the user's preferences and past reviews. These suggestions include price comparisons and restaurant location information.

[0556] Finally, the terminal displays the suggestions received from the server to the user. The user makes a decision based on this information and sends the result and feedback back to the server via the terminal. The server analyzes this feedback information and uses it to improve the accuracy of the suggestions. This adjusts future suggestions to be more accurate and user-oriented.

[0557] This embodiment allows users to intuitively and efficiently obtain the necessary information and make decisions without having to perform cumbersome information searches.

[0558] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0559] Step 1:

[0560] The user uses an information device to input specific purposes or wishes in natural language. The input text is received on the user's device, and a prompt message such as "Please recommend some restaurants for the weekend" is generated. The input natural language text becomes the basic data for subsequent processing.

[0561] Step 2:

[0562] The terminal sends user input to the server. During this process, the input data is securely encrypted using the SSL / TLS protocol and transmitted. The transmitted prompt message is awaiting analysis by the server.

[0563] Step 3:

[0564] The server analyzes the received natural language input using a generative AI model. Specifically, it uses natural language processing techniques to identify the intent of the input sentence and extract keywords. The input is a prompt sentence, and the output is a set of analyzed keywords and themes (e.g., "restaurant," "weekend," etc.).

[0565] Step 4:

[0566] The server collects relevant data from the internet based on the analyzed keywords. Here, web scraping tools are used to gather specific store information, price information, and other data from publicly available information and databases on the internet. The input is keywords, and the output is a collection of relevant data.

[0567] Step 5:

[0568] The server processes the collected data using data analysis techniques to generate personalized recommendations for the user. For example, it compares price information from a database to suggest the most cost-effective restaurant. It also includes personalized advice based on past user preferences. The input is the collected raw data, and the output is specific recommendation information.

[0569] Step 6:

[0570] The proposed information is sent from the server to the terminal. This transmission involves formatting the data (e.g., JSON format) to ensure it is suitable for display. The input is the proposed information, and the output is the data sent to the user's terminal.

[0571] Step 7:

[0572] The device presents the user with appropriately formatted suggestion information. The user can view this information and, if necessary, request further details using the voice assistant or touch interface. The input is formatted data, and the output is visual or audible feedback to the user.

[0573] Step 8:

[0574] Users make decisions based on the information presented and provide feedback on the results and their impressions through their device. This feedback is used to improve future proposals. The input is the user's experience and choices, and the output is feedback data.

[0575] Step 9:

[0576] The server receives feedback data and uses machine learning techniques to improve the system's proposed algorithm. This process enables more accurate personalized recommendations for subsequent uses. The input is feedback data, and the output is the improved algorithm model.

[0577] (Application Example 1)

[0578] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0579] When users choose products and services in their daily lives, it is difficult for them to quickly and accurately find the best option from a vast amount of information. Furthermore, effectively utilizing payment options and discount information is also challenging. This complicates the user's purchasing experience and hinders optimal decision-making.

[0580] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0581] In this invention, the server includes means for analyzing user input using natural language processing to identify the user's intent, means for generating optimal suggestions for the user using the collected data, and means for providing the user with optimal payment options and discount information in real time. This enables users to efficiently obtain relevant information and quickly make optimal choices in their daily purchasing activities.

[0582] "User input" refers to the act of a user sending information to a system via a terminal, and the content of that information, including natural language and instructions.

[0583] "Natural language processing" is a technology that uses computers to analyze human language and understand user intent and information.

[0584] "Automatically collecting relevant data from the internet" refers to the process of automatically searching for and retrieving relevant information that exists on a network.

[0585] "Generating suggestions" is the act of organizing and presenting the most suitable options and information for the user based on collected data.

[0586] "User terminal" refers to a digital device used by a user to receive information, and includes smartphones, computers, and other similar devices.

[0587] "Improving the accuracy of suggestions based on feedback" is a process of analyzing user responses and results to improve the quality of information provided in the future.

[0588] "Payment options" refer to the payment methods and conditions available when purchasing goods or services.

[0589] "Discount information" refers to information about price reductions or benefits that apply under specific conditions.

[0590] The system for realizing this invention consists of a user-owned digital device (such as a smartphone or smart glasses) and a server connected via the internet. The server receives user input, performs natural language processing, and analyzes the user's intentions and requests. This analysis uses natural language processing libraries (e.g., spaCy, Google NLP API).

[0591] The server automatically collects information on relevant products and services from the internet based on the analysis results. Web scraping tools (e.g., BeautifulSoup, Selenium) are used for this information collection. The collected data is then used to generate suggestions best suited to the user's needs. Generative AI models may be used to personalize these suggestions.

[0592] The user's device displays suggested information received from the server via voice and visuals. The user reviews this information and provides additional instructions as needed, such as through voice input. Furthermore, payment options and discount information related to the products and services the user is considering are also provided in real time.

[0593] As a concrete example, consider a scenario where a user is planning their grocery shopping trip using smart glasses. If the user voice-inputs, "I want to know what the cheapest detergent is right now," the system analyzes the information and immediately displays the latest price information and available discounts.

[0594] By utilizing generative AI models in this process, it becomes possible to provide optimal suggestions to users quickly and with high accuracy, improving the user's purchasing experience. It also accepts prompt messages such as, "I want to buy the ingredients I need to prepare tonight's dinner at a good price. Tell me about coupons I can use at the supermarket I'm planning to go to and the best payment method," and provides appropriate information immediately.

[0595] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0596] Step 1:

[0597] The user inputs information using natural language via a smartphone or smart glasses. This input is sent to the server. A natural language processing engine analyzes the input text to identify the user's intent and desires. This process utilizes natural language processing to break down the text and extract keywords and requirements. For example, if a request includes "cheapest detergent," the keywords "cheapest" and "detergent" are identified.

[0598] Step 2:

[0599] The server collects relevant data from the internet based on the identified keywords. Here, web scraping techniques are used to obtain price and discount information from various websites. This method efficiently collects the latest data that matches the user's requirements. The collected data is stored in a database and used for analysis.

[0600] Step 3:

[0601] The server analyzes the collected data and generates the most suitable suggestions for the user. Using a generative AI model, it builds personalized suggestions based on the collected information, including price comparisons of products and the user's purchase history. This analysis generates suggestions for the most suitable products, payment options, and discount information for the user.

[0602] Step 4:

[0603] The generated suggestions are sent from the server to the user's terminal. The terminal presents this information visually or audibly through its user interface. This allows the user to easily review the suggestions. The terminal formats the received data and displays it in a way that is easy for the user to understand.

[0604] Step 5:

[0605] Users can retrieve the presented information and provide feedback on it. This feedback is sent to the server via the device. The server uses the collected feedback to improve the accuracy of suggestions and personalization through algorithmic processing. Machine learning is performed based on the feedback to improve the quality of information provided in the future.

[0606] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0607] This invention is an information provision system that takes into account the user's emotional state. The system aims to provide more personalized suggestions by analyzing the user's natural language input and simultaneously evaluating the user's intentions and emotions.

[0608] 1. Processing user input and emotion recognition

[0609] Users input requests and objectives into the device using natural language, and their emotional state is collected based on their tone of voice, facial expressions, and input content. The device then sends this data to a server.

[0610] 2. Utilization of Natural Language Processing and Sentiment Engines

[0611] The server analyzes the user's input data using natural language processing techniques to identify the user's intent and recognizes the user's emotional state using an emotion engine. The emotional information obtained in this step is used to customize the suggested content.

[0612] 3. Collection and analysis of relevant data

[0613] The server collects necessary data from the internet based on the user's intent and analyzes it using AI technology. This data collection includes product information, price comparisons, and service evaluations.

[0614] 4. Personalizing suggestions

[0615] The server comprehensively considers the acquired data and the user's emotional state to generate the most suitable suggestions for the user. These suggestions include a message tone and content that reflects the user's emotions.

[0616] 5. Information presentation and interface optimization

[0617] The terminal displays suggestions received from the server on the user interface. During this process, the interface's color scheme and message tone are adjusted based on the user's emotions. For example, if the user is feeling stressed, calm colors and gentle language will be used.

[0618] 6. Gathering feedback and further learning

[0619] When a user accepts a suggestion and inputs the result as feedback on their device, the system learns further. Based on the feedback, the server updates the model to improve sentiment recognition and suggestion accuracy.

[0620] For example, if a user enters "I'm looking for relaxation items to relieve stress on my days off," the system will sense fatigue from their voice. Based on this, it will generate suggestions, including products with high relaxation effects and special offers for comfortable delivery services, which will be presented to the user in a gentle tone through their device.

[0621] This system allows users to receive suggestions that best suit their emotional state, resulting in a more satisfying experience.

[0622] The following describes the processing flow.

[0623] Step 1:

[0624] Users input the information and purpose they are looking for into the device using natural language. Simultaneously, the user's voice tone and facial expressions are collected as emotional data.

[0625] Step 2:

[0626] The device sends the user's natural language input and associated sentiment data to the server. This data forms the basis for the system to understand the user's needs and emotional state.

[0627] Step 3:

[0628] The server analyzes the received data using natural language processing techniques to identify the user's intent. Simultaneously, it uses an emotion engine to analyze the user's emotional state.

[0629] Step 4:

[0630] The server collects relevant data from the internet based on the identified user's intentions and emotional state. This data can range from product information and service details to ratings and price comparisons.

[0631] Step 5:

[0632] The server uses AI technology to analyze the collected data and generate suggestions that are best suited to the user's intentions and emotional state. This process is highly sensitive to changes in emotions and dynamically adjusts the suggestions accordingly.

[0633] Step 6:

[0634] The server sends the generated suggestions to the terminal. The suggestions include a message tone that suits the user's emotions and highlights the benefits of specific products.

[0635] Step 7:

[0636] The device presents suggestions to the user through its interface. The screen's color scheme and message tone may change depending on the user's mood.

[0637] Step 8:

[0638] The user reviews the presented suggestions and selects an action. Based on this selection, the user can input feedback on the suggestions into the device.

[0639] Step 9:

[0640] The device sends user feedback to the server. This allows the entire system to learn and contributes to improving the accuracy of future suggestions.

[0641] Step 10:

[0642] The server updates its sentiment recognition model and suggestion generation algorithm based on the collected feedback. This allows for more precise and user-friendly suggestions.

[0643] (Example 2)

[0644] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0645] Modern information delivery systems generally only provide one-way information in response to user requests, lacking personalization that takes into account the user's emotional state. As a result, the user experience is unsatisfactory, and it is difficult to meet the diverse needs of users. This invention aims to solve these problems and provide more comprehensive information that simultaneously considers the user's intentions and emotional state.

[0646] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0647] In this invention, the server includes means for analyzing user input using natural language processing technology to identify the user's intent, means for recognizing the user's emotional state based on voice tone and facial expression data acquired during user input, and means for automatically collecting and analyzing relevant information from the internet. This makes it possible to provide the user with optimal information that takes into account the user's intent and emotional state, thereby providing a more personalized and satisfying experience.

[0648] "User input" refers to requests, objectives, and information that a user transmits to the system via a terminal.

[0649] "Natural language processing technology" is a technology that enables computers to analyze and understand human language.

[0650] "Identifying intent" refers to accurately understanding the user's requests and objectives based on their input.

[0651] "Voice tone" refers to the pitch, volume, and overall tone of voice used to express emotions when a user speaks.

[0652] "Facial expression data" refers to information that records changes in a user's face, indicating their emotions and reactions.

[0653] "Emotional state" refers to the psychological state or emotional characteristics that a user exhibits when entering information.

[0654] "Related information" refers to data and information related to the user's requests and objectives, and is obtained from the internet or databases.

[0655] "Collecting from the internet" means obtaining necessary data from online sources.

[0656] "Analyzing" is the act of examining collected data to find the desired meaning and relationships.

[0657] "Optimal information" refers to information that provides the most suitable suggestions and answers to the user, based on the user's intentions and emotional state.

[0658] "Feedback" refers to a user's reaction to or evaluation of suggestions or information provided.

[0659] "Learning" is the process by which a system improves itself based on collected feedback and other factors.

[0660] A "user terminal" is a computer graphics or mobile device used by a user to input information or receive results.

[0661] This system incorporates technology designed to provide information that takes into account the user's emotional state. It begins with the user entering a request in natural language into a terminal. In addition to the input, the terminal collects supplementary information such as voice tone and facial expression data. This collected data is then transmitted to a server via secure communication (e.g., TLS protocol).

[0662] The server uses natural language processing technology to analyze user input and identify the user's intent. A generative AI model is used, and the prompt "Analyze the user's request and identify their intent" is input. Simultaneously, the server uses an emotion engine to recognize the emotional state from voice tone and facial expression data.

[0663] Relevant information is automatically collected via the internet and internal databases, and the necessary data is analyzed using AI technology. This process utilizes product information APIs, among others. Based on the collected information and recognized emotional states, the server generates information best suited to the user and creates personalized suggestions.

[0664] The suggestions are then sent to the user's terminal and displayed on the user interface with appropriate colors and tones. This allows the user to receive information optimized according to their emotional state. For example, if a user enters "I'm looking for relaxation items to relieve stress on my days off," the system will suggest relaxation products based on the user's emotions and display the information in a calming tone.

[0665] Finally, the user inputs feedback on the suggested information into the device. The server then uses this feedback to collect data and update the AI ​​model to improve the accuracy of suggestions and sentiment recognition. By repeating this process, the system is continuously improved, enhancing the quality of the user experience.

[0666] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0667] Step 1:

[0668] The user inputs information into the device using natural language. The device records the user's requests and objectives as text data, while simultaneously collecting voice tone and facial expression data using the microphone and camera. All of this input data is organized and prepared as a single data package for the next processing step. At this stage, the input content is output as text data, and accompanying information is output as voice and facial expression data.

[0669] Step 2:

[0670] The terminal sends the organized data package to the server. Data security is ensured using protocols such as TLS. The server receives the transmitted data and prepares for the next analysis step. The input is the data package from the terminal, and the output is the data converted into an analyzable format.

[0671] Step 3:

[0672] The server uses natural language processing techniques to analyze text data and identify the user's intent. Here, the generative AI model is input with the prompt, "Analyze the user's request and identify their intent." The server then outputs keywords and objective information related to the user's intent as part of its analysis.

[0673] Step 4:

[0674] The server uses an emotion engine to recognize the user's emotional state from voice tone and facial expression data. Voice analysis algorithms and facial recognition technology are used. Input is voice and facial expression data, and output is tags and numerical indicators that show emotional tendencies.

[0675] Step 5:

[0676] The server collects relevant information from the internet and databases based on the user's intent and emotional state. This process involves the execution of product information APIs and online search algorithms. Input consists of keywords and intent information, while output is a list of relevant information.

[0677] Step 6:

[0678] The server generates information best suited to the user based on collected data and recognized emotional states. An AI model analyzes the data and determines the most appropriate suggestions. The input is collected information and emotional data, and the output is personalized suggestions.

[0679] Step 7:

[0680] The device displays the received suggestions on the user interface. The tone and color scheme of the suggestions are adjusted according to the user's emotional state. The input is the suggested content, and the output is the visual presentation of the information.

[0681] Step 8:

[0682] The user provides feedback on the presented information. The device records this feedback and sends it to the server. This data is used to improve the accuracy of future suggestions. The input is the user's comments and evaluations, and the output is the recorded feedback data.

[0683] Step 9:

[0684] The server analyzes the feedback and updates the AI ​​model. This process continuously improves sentiment recognition and suggestion accuracy. The input is feedback data, and the output is the updated AI model.

[0685] (Application Example 2)

[0686] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0687] Conventional information delivery systems have difficulty providing personalized suggestions that take into account the user's emotional state, resulting in challenges in improving user satisfaction. Furthermore, because the suggested content is not optimized for the user's emotional state at the time, it has been difficult to improve the user experience.

[0688] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0689] In this invention, the server includes means for analyzing user input using natural language processing to simultaneously identify the user's intent and emotional state, means for acquiring audio and video data to recognize the user's emotional state, and means for automatically collecting relevant information from the internet and analyzing it using artificial intelligence technology. This makes it possible to provide personalized suggestions that are optimal for the user's emotional state.

[0690] "User input" refers to the expression of requests or objectives made by a user through a device, either via voice or text.

[0691] "Natural language processing" is the technology that enables computers to understand, analyze, and process human language.

[0692] "Intention" refers to the purpose or wish that the user is trying to convey through their input.

[0693] "Emotional state" refers to information that indicates the emotions a user is feeling at a particular point in time.

[0694] "Audio data" refers to audio information that records the user's voice.

[0695] "Video data" refers to video information that records the user's facial expressions and movements.

[0696] "Automatically collecting relevant information from the internet" refers to the process of automatically retrieving information that is publicly available online.

[0697] "Artificial intelligence technology" is a technology that allows computers to mimic some aspects of human intelligence.

[0698] A "user interface" refers to the screens and methods of operation that a user uses when interacting with a system.

[0699] "Feedback" refers to the opinions and responses that users provide in response to suggestions they receive.

[0700] "Proposal accuracy" is an indicator of how well the proposals are tailored to the user's needs and feelings.

[0701] The system for realizing this invention uses a user terminal such as a smartphone or tablet and a server in the cloud. When the user inputs voice or text into the terminal, the terminal uses its built-in microphone and camera to collect data on the user's voice and facial expressions. This information is transmitted to the server via the internet.

[0702] On the server, natural language processing software (e.g., Google Cloud Natural Language API) is first used to analyze the user's input text and identify the user's intent. Simultaneously, an emotion recognition engine recognizes emotional states from audio and video data. This analysis utilizes speech recognition APIs (e.g., Google Cloud Speech-to-Text API) and facial expression recognition libraries (e.g., OpenCV).

[0703] Based on the user's intent and emotional state derived from this data, the server collects relevant information from the internet and analyzes it using artificial intelligence technology. The analysis utilizes machine learning models to generate personalized suggestions best suited to the user. These suggestions are sent to the user's device, and the content displayed in the user interface is adjusted in terms of color scheme and message tone to match the user's emotional state.

[0704] For example, if a user intends to relax on their day off and the server detects stress from their tone of voice, it will suggest products with relaxing effects and comfortable delivery options in a gentle tone.

[0705] An example of an input prompt for a generative AI model could be, "Please create a personalized suggestion that considers products and benefits to a user who wants to relax." Based on this prompt, the AI ​​can generate optimal suggestions that match the user's emotions.

[0706] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0707] Step 1:

[0708] Users input information into their smartphones or tablets via voice or text. The device receives this input and simultaneously captures audio and video data using its built-in microphone and camera. This collects data about the user's intent and emotional data based on voice tone and facial expressions.

[0709] Step 2:

[0710] The device converts the collected audio data into text using a speech recognition API and sends it to the server. At the same time, video data is also sent for emotion recognition and used as material to evaluate the user's emotional state from their facial expressions. This forms the input dataset.

[0711] Step 3:

[0712] The server analyzes the transcribed user input using natural language processing techniques to identify the user's specific intentions. It also uses an emotion engine to recognize emotional states based on audio and video data. The input data consists of speech recognition results and video data, while the output provides metadata of the user's intentions and emotions.

[0713] Step 4:

[0714] The server collects relevant information from the internet based on the analyzed intent and sentiment metadata. It uses AI technology to analyze product information and reviews and generate personalized suggestions for the user. In this process, the input relevant information is processed into personalized suggestions as output.

[0715] Step 5:

[0716] The generated suggestions are sent to the device to be presented with a tone and color scheme adjusted based on the user's emotional state. The device then appropriately displays the received suggestions on the user interface. At this time, the displayed content will have its color scheme and tone adjusted based on the input emotional metadata.

[0717] Step 6:

[0718] The user sends feedback on the presented suggestions from their device to the server. The server collects this feedback data as training material and uses it to improve the accuracy of the suggestions and the sentiment recognition model. This feedback loop improves the performance of the output system in subsequent uses.

[0719] For the generating AI model, the prompt "Please create a personalized suggestion that considers products and benefits to recommend when the user is feeling relaxed" is used to generate AI-based suggestions.

[0720] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0721] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0722] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0723] [Fourth Embodiment]

[0724] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0725] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0726] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0727] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0728] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0729] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0730] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0731] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0732] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0733] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0734] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0735] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0736] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0737] This invention is a system that supports users in efficiently acquiring information and making optimal decisions in their daily lives. The system operates with its components working together as follows:

[0738] 1. Processing user input

[0739] Users input their wishes and objectives in natural language via their devices. For example, "I want to have a party this weekend, so I'd like information on ingredients and venues."

[0740] 2. Analysis using natural language processing

[0741] The server receives user input sent from the terminal and analyzes it using natural language processing technology. This analysis identifies the user's intent and the information they need.

[0742] 3. Collection of related data

[0743] The server collects necessary data from the internet based on the analysis results. Specifically, it acquires website data to obtain information about products and services that users are looking for, as well as the best prices.

[0744] 4. Proposal generation

[0745] The server analyzes the collected data and generates suggestions that best meet the user's needs. This includes price comparisons, location guidance, and coupon information.

[0746] 5. Presenting information to the user

[0747] The terminal displays suggested information received from the server to the user. The user can also ask additional questions using the voice assistant.

[0748] 6. Gathering and Learning from Feedback

[0749] The user acts based on the presented suggestion. The device records the result and sends it to the server. The server learns from this feedback and uses it as data to improve future suggestions.

[0750] As a concrete example, if a user is planning to purchase daily necessities, the system can be used as follows: When the user inputs "I want to buy detergent and shampoo by this weekend," the server analyzes the request, compares information on nearby stores and online prices, and suggests the best purchase options. Furthermore, the server can refer to the user's past purchase history and recommend their favorite brands. The terminal presents this information to the user, who then makes a purchase based on the suggestions. The user's purchase behavior is sent to the server as feedback and used to improve the accuracy of future suggestions.

[0751] This configuration allows users to obtain information efficiently and without hassle, enabling them to make optimal choices.

[0752] The following describes the processing flow.

[0753] Step 1:

[0754] The user inputs their wishes and requests related to their daily life into the terminal using natural language. This input serves as the initial trigger for this system.

[0755] Step 2:

[0756] The terminal transmits user input as digital data to the server. This data is necessary to analyze the user's intent.

[0757] Step 3:

[0758] The server uses natural language processing techniques to analyze user input. This analysis specifically identifies the user's requests and objectives, and determines the type of information needed.

[0759] Step 4:

[0760] The server collects relevant data from the internet based on the analysis results. This data collection is performed through API access and web scraping.

[0761] Step 5:

[0762] The server uses AI technology to analyze the collected data and generate optimal recommendations for the user. This analysis includes price comparisons, quality evaluations, and user reviews.

[0763] Step 6:

[0764] The server sends the generated suggestion to the terminal. This suggestion is a structured version of the information the user is looking for.

[0765] Step 7:

[0766] The terminal displays suggestions from the server to the user via a user interface. The user can then ask more detailed questions by activating the voice assistant.

[0767] Step 8:

[0768] The user makes decisions based on the suggestions presented on the device. For example, they might choose to make a purchase at a suggested store.

[0769] Step 9:

[0770] After a user's action, the device records the result and sends the user's feedback to the server.

[0771] Step 10:

[0772] The server uses feedback to update the AI ​​model and learn to improve the accuracy of future suggestions.

[0773] (Example 1)

[0774] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0775] In modern society, there are limited means to efficiently gather the information users need and support them in making optimal decisions. In particular, in situations where quick and appropriate information acquisition is required in daily activities, users find it difficult to discern the best option from many choices. Furthermore, in an information-overload environment, mechanisms for providing personalized suggestions based on individual needs and preferences are not yet fully established.

[0776] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0777] In this invention, the server includes means for analyzing natural language input received from a user via an information device to identify the user's intent, means for automatically collecting and analyzing relevant information from a database on the Internet using an information processing device, and means for generating suggestions suitable for the user's request based on the collected information. This makes it possible to obtain optimal information and provide suggestions based on the user's intent.

[0778] "Information equipment" is a general term for terminals and devices that users use to input information.

[0779] "Natural language input" is a method by which users provide information to information devices using natural language and sentences.

[0780] An "information processing device" is a computing device used to collect, analyze, and process digital data.

[0781] A "database" is a structured collection of information used to systematically store and access necessary information.

[0782] "Relevant information" refers to data and facts related to the user's intentions and requests.

[0783] "Proposal" means presenting users with convenient and optimal options or actions based on the information collected.

[0784] "Feedback information" refers to information about the results and reactions related to user actions and choices.

[0785] "Personalized suggestions" refer to suggestions that are specially customized to match the user's past behavior history and preferences.

[0786] This invention is a system that supports the efficient acquisition of information and the optimization of decision-making in the user's daily life. This system is implemented using information equipment, servers, and related software technologies.

[0787] First, the user inputs their purpose or request in natural language via an information device. For example, they might input a request such as, "I'm looking for a place to meet up with friends this weekend, and I'd like to know the best restaurant." This input is received by the terminal, and the user's response is sent to the server as a prompt. An example of such a prompt would be, "Please recommend a restaurant for the weekend."

[0788] Next, the server utilizes a generative AI model and natural language processing techniques to analyze user input. Through this analysis, the server identifies the user's intent and, based on that information, collects relevant data from the internet. This data collection can be done using database queries or web scraping techniques. Specifically, AI technologies known as generative AI models and web scraping tools are used.

[0789] The server then processes the collected information using advanced data analysis techniques to generate optimal suggestions for the user. For example, based on the collected restaurant information, it provides personalized suggestions based on the user's preferences and past reviews. These suggestions include price comparisons and restaurant location information.

[0790] Finally, the terminal displays the suggestions received from the server to the user. The user makes a decision based on this information and sends the result and feedback back to the server via the terminal. The server analyzes this feedback information and uses it to improve the accuracy of the suggestions. This adjusts future suggestions to be more accurate and user-oriented.

[0791] This embodiment allows users to intuitively and efficiently obtain the necessary information and make decisions without having to perform cumbersome information searches.

[0792] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0793] Step 1:

[0794] The user uses an information device to input specific purposes or wishes in natural language. The input text is received on the user's device, and a prompt message such as "Please recommend some restaurants for the weekend" is generated. The input natural language text becomes the basic data for subsequent processing.

[0795] Step 2:

[0796] The terminal sends user input to the server. During this process, the input data is securely encrypted using the SSL / TLS protocol and transmitted. The transmitted prompt message is awaiting analysis by the server.

[0797] Step 3:

[0798] The server analyzes the received natural language input using a generative AI model. Specifically, it uses natural language processing techniques to identify the intent of the input sentence and extract keywords. The input is a prompt sentence, and the output is a set of analyzed keywords and themes (e.g., "restaurant," "weekend," etc.).

[0799] Step 4:

[0800] The server collects relevant data from the internet based on the analyzed keywords. Here, web scraping tools are used to gather specific store information, price information, and other data from publicly available information and databases on the internet. The input is keywords, and the output is a collection of relevant data.

[0801] Step 5:

[0802] The server processes the collected data using data analysis techniques to generate personalized recommendations for the user. For example, it compares price information from a database to suggest the most cost-effective restaurant. It also includes personalized advice based on past user preferences. The input is the collected raw data, and the output is specific recommendation information.

[0803] Step 6:

[0804] The proposed information is sent from the server to the terminal. This transmission involves formatting the data (e.g., JSON format) to ensure it is suitable for display. The input is the proposed information, and the output is the data sent to the user's terminal.

[0805] Step 7:

[0806] The device presents the user with appropriately formatted suggestion information. The user can view this information and, if necessary, request further details using the voice assistant or touch interface. The input is formatted data, and the output is visual or audible feedback to the user.

[0807] Step 8:

[0808] Users make decisions based on the information presented and provide feedback on the results and their impressions through their device. This feedback is used to improve future proposals. The input is the user's experience and choices, and the output is feedback data.

[0809] Step 9:

[0810] The server receives feedback data and uses machine learning techniques to improve the system's proposed algorithm. This process enables more accurate personalized recommendations for subsequent uses. The input is feedback data, and the output is the improved algorithm model.

[0811] (Application Example 1)

[0812] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0813] When users choose products and services in their daily lives, it is difficult for them to quickly and accurately find the best option from a vast amount of information. Furthermore, effectively utilizing payment options and discount information is also challenging. This complicates the user's purchasing experience and hinders optimal decision-making.

[0814] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0815] In this invention, the server includes means for analyzing user input using natural language processing to identify the user's intent, means for generating optimal suggestions for the user using the collected data, and means for providing the user with optimal payment options and discount information in real time. This enables users to efficiently obtain relevant information and quickly make optimal choices in their daily purchasing activities.

[0816] "User input" refers to the act of a user sending information to a system via a terminal, and the content of that information, including natural language and instructions.

[0817] "Natural language processing" is a technology that uses computers to analyze human language and understand user intent and information.

[0818] "Automatically collecting relevant data from the internet" refers to the process of automatically searching for and retrieving relevant information that exists on a network.

[0819] "Generating suggestions" is the act of organizing and presenting the most suitable options and information for the user based on collected data.

[0820] "User terminal" refers to a digital device used by a user to receive information, and includes smartphones, computers, and other similar devices.

[0821] "Improving the accuracy of suggestions based on feedback" is a process of analyzing user responses and results to improve the quality of information provided in the future.

[0822] "Payment options" refer to the payment methods and conditions available when purchasing goods or services.

[0823] "Discount information" refers to information about price reductions or benefits that apply under specific conditions.

[0824] The system for realizing this invention consists of a user-owned digital device (such as a smartphone or smart glasses) and a server connected via the internet. The server receives user input, performs natural language processing, and analyzes the user's intentions and requests. This analysis uses natural language processing libraries (e.g., spaCy, Google NLP API).

[0825] The server automatically collects information on relevant products and services from the internet based on the analysis results. Web scraping tools (e.g., BeautifulSoup, Selenium) are used for this information collection. The collected data is then used to generate suggestions best suited to the user's needs. Generative AI models may be used to personalize these suggestions.

[0826] The user's device displays suggested information received from the server via voice and visuals. The user reviews this information and provides additional instructions as needed, such as through voice input. Furthermore, payment options and discount information related to the products and services the user is considering are also provided in real time.

[0827] As a concrete example, consider a scenario where a user is planning their grocery shopping trip using smart glasses. If the user voice-inputs, "I want to know what the cheapest detergent is right now," the system analyzes the information and immediately displays the latest price information and available discounts.

[0828] By utilizing generative AI models in this process, it becomes possible to provide optimal suggestions to users quickly and with high accuracy, improving the user's purchasing experience. It also accepts prompt messages such as, "I want to buy the ingredients I need to prepare tonight's dinner at a good price. Tell me about coupons I can use at the supermarket I'm planning to go to and the best payment method," and provides appropriate information immediately.

[0829] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0830] Step 1:

[0831] The user inputs information using natural language via a smartphone or smart glasses. This input is sent to the server. A natural language processing engine analyzes the input text to identify the user's intent and desires. This process utilizes natural language processing to break down the text and extract keywords and requirements. For example, if a request includes "cheapest detergent," the keywords "cheapest" and "detergent" are identified.

[0832] Step 2:

[0833] The server collects relevant data from the internet based on the identified keywords. Here, web scraping techniques are used to obtain price and discount information from various websites. This method efficiently collects the latest data that matches the user's requirements. The collected data is stored in a database and used for analysis.

[0834] Step 3:

[0835] The server analyzes the collected data and generates the most suitable suggestions for the user. Using a generative AI model, it builds personalized suggestions based on the collected information, including price comparisons of products and the user's purchase history. This analysis generates suggestions for the most suitable products, payment options, and discount information for the user.

[0836] Step 4:

[0837] The generated suggestions are sent from the server to the user's terminal. The terminal presents this information visually or audibly through its user interface. This allows the user to easily review the suggestions. The terminal formats the received data and displays it in a way that is easy for the user to understand.

[0838] Step 5:

[0839] Users can retrieve the presented information and provide feedback on it. This feedback is sent to the server via the device. The server uses the collected feedback to improve the accuracy of suggestions and personalization through algorithmic processing. Machine learning is performed based on the feedback to improve the quality of information provided in the future.

[0840] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0841] This invention is an information provision system that takes into account the user's emotional state. The system aims to provide more personalized suggestions by analyzing the user's natural language input and simultaneously evaluating the user's intentions and emotions.

[0842] 1. Processing user input and emotion recognition

[0843] Users input requests and objectives into the device using natural language, and their emotional state is collected based on their tone of voice, facial expressions, and input content. The device then sends this data to a server.

[0844] 2. Utilization of Natural Language Processing and Sentiment Engines

[0845] The server analyzes the user's input data using natural language processing techniques to identify the user's intent and recognizes the user's emotional state using an emotion engine. The emotional information obtained in this step is used to customize the suggested content.

[0846] 3. Collection and analysis of relevant data

[0847] The server collects necessary data from the internet based on the user's intent and analyzes it using AI technology. This data collection includes product information, price comparisons, and service evaluations.

[0848] 4. Personalizing suggestions

[0849] The server comprehensively considers the acquired data and the user's emotional state to generate the most suitable suggestions for the user. These suggestions include a message tone and content that reflects the user's emotions.

[0850] 5. Information presentation and interface optimization

[0851] The terminal displays suggestions received from the server on the user interface. During this process, the interface's color scheme and message tone are adjusted based on the user's emotions. For example, if the user is feeling stressed, calm colors and gentle language will be used.

[0852] 6. Gathering feedback and further learning

[0853] When a user accepts a suggestion and inputs the result as feedback on their device, the system learns further. Based on the feedback, the server updates the model to improve sentiment recognition and suggestion accuracy.

[0854] For example, if a user enters "I'm looking for relaxation items to relieve stress on my days off," the system will sense fatigue from their voice. Based on this, it will generate suggestions, including products with high relaxation effects and special offers for comfortable delivery services, which will be presented to the user in a gentle tone through their device.

[0855] This system allows users to receive suggestions that best suit their emotional state, resulting in a more satisfying experience.

[0856] The following describes the processing flow.

[0857] Step 1:

[0858] Users input the information and purpose they are looking for into the device using natural language. Simultaneously, the user's voice tone and facial expressions are collected as emotional data.

[0859] Step 2:

[0860] The device sends the user's natural language input and associated sentiment data to the server. This data forms the basis for the system to understand the user's needs and emotional state.

[0861] Step 3:

[0862] The server analyzes the received data using natural language processing techniques to identify the user's intent. Simultaneously, it uses an emotion engine to analyze the user's emotional state.

[0863] Step 4:

[0864] The server collects relevant data from the internet based on the identified user's intentions and emotional state. This data can range from product information and service details to ratings and price comparisons.

[0865] Step 5:

[0866] The server uses AI technology to analyze the collected data and generate suggestions that are best suited to the user's intentions and emotional state. This process is highly sensitive to changes in emotions and dynamically adjusts the suggestions accordingly.

[0867] Step 6:

[0868] The server sends the generated suggestions to the terminal. The suggestions include a message tone that suits the user's emotions and highlights the benefits of specific products.

[0869] Step 7:

[0870] The device presents suggestions to the user through its interface. The screen's color scheme and message tone may change depending on the user's mood.

[0871] Step 8:

[0872] The user reviews the presented suggestions and selects an action. Based on this selection, the user can input feedback on the suggestions into the device.

[0873] Step 9:

[0874] The device sends user feedback to the server. This allows the entire system to learn and contributes to improving the accuracy of future suggestions.

[0875] Step 10:

[0876] The server updates its sentiment recognition model and suggestion generation algorithm based on the collected feedback. This allows for more precise and user-friendly suggestions.

[0877] (Example 2)

[0878] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0879] Modern information delivery systems generally only provide one-way information in response to user requests, lacking personalization that takes into account the user's emotional state. As a result, the user experience is unsatisfactory, and it is difficult to meet the diverse needs of users. This invention aims to solve these problems and provide more comprehensive information that simultaneously considers the user's intentions and emotional state.

[0880] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0881] In this invention, the server includes means for analyzing user input using natural language processing technology to identify the user's intent, means for recognizing the user's emotional state based on voice tone and facial expression data acquired during user input, and means for automatically collecting and analyzing relevant information from the internet. This makes it possible to provide the user with optimal information that takes into account the user's intent and emotional state, thereby providing a more personalized and satisfying experience.

[0882] "User input" refers to requests, objectives, and information that a user transmits to the system via a terminal.

[0883] "Natural language processing technology" is a technology that enables computers to analyze and understand human language.

[0884] "Identifying intent" refers to accurately understanding the user's requests and objectives based on their input.

[0885] "Voice tone" refers to the pitch, volume, and overall tone of voice used to express emotions when a user speaks.

[0886] "Facial expression data" refers to information that records changes in a user's face, indicating their emotions and reactions.

[0887] "Emotional state" refers to the psychological state or emotional characteristics that a user exhibits when entering information.

[0888] "Related information" refers to data and information related to the user's requests and objectives, and is obtained from the internet or databases.

[0889] "Collecting from the internet" means obtaining necessary data from online sources.

[0890] "Analyzing" is the act of examining collected data to find the desired meaning and relationships.

[0891] "Optimal information" refers to information that provides the most suitable suggestions and answers to the user, based on the user's intentions and emotional state.

[0892] "Feedback" refers to a user's reaction to or evaluation of suggestions or information provided.

[0893] "Learning" is the process by which a system improves itself based on collected feedback and other factors.

[0894] A "user terminal" is a computer graphics or mobile device used by a user to input information or receive results.

[0895] This system incorporates technology designed to provide information that takes into account the user's emotional state. It begins with the user entering a request in natural language into a terminal. In addition to the input, the terminal collects supplementary information such as voice tone and facial expression data. This collected data is then transmitted to a server via secure communication (e.g., TLS protocol).

[0896] The server uses natural language processing technology to analyze user input and identify the user's intent. A generative AI model is used, and the prompt "Analyze the user's request and identify their intent" is input. Simultaneously, the server uses an emotion engine to recognize the emotional state from voice tone and facial expression data.

[0897] Relevant information is automatically collected via the internet and internal databases, and the necessary data is analyzed using AI technology. This process utilizes product information APIs, among others. Based on the collected information and recognized emotional states, the server generates information best suited to the user and creates personalized suggestions.

[0898] The suggestions are then sent to the user's terminal and displayed on the user interface with appropriate colors and tones. This allows the user to receive information optimized according to their emotional state. For example, if a user enters "I'm looking for relaxation items to relieve stress on my days off," the system will suggest relaxation products based on the user's emotions and display the information in a calming tone.

[0899] Finally, the user inputs feedback on the suggested information into the device. The server then uses this feedback to collect data and update the AI ​​model to improve the accuracy of suggestions and sentiment recognition. By repeating this process, the system is continuously improved, enhancing the quality of the user experience.

[0900] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0901] Step 1:

[0902] The user inputs information into the device using natural language. The device records the user's requests and objectives as text data, while simultaneously collecting voice tone and facial expression data using the microphone and camera. All of this input data is organized and prepared as a single data package for the next processing step. At this stage, the input content is output as text data, and accompanying information is output as voice and facial expression data.

[0903] Step 2:

[0904] The terminal sends the organized data package to the server. Data security is ensured using protocols such as TLS. The server receives the transmitted data and prepares for the next analysis step. The input is the data package from the terminal, and the output is the data converted into an analyzable format.

[0905] Step 3:

[0906] The server uses natural language processing techniques to analyze text data and identify the user's intent. Here, the generative AI model is input with the prompt, "Analyze the user's request and identify their intent." The server then outputs keywords and objective information related to the user's intent as part of its analysis.

[0907] Step 4:

[0908] The server uses an emotion engine to recognize the user's emotional state from voice tone and facial expression data. Voice analysis algorithms and facial recognition technology are used. Input is voice and facial expression data, and output is tags and numerical indicators that show emotional tendencies.

[0909] Step 5:

[0910] The server collects relevant information from the internet and databases based on the user's intent and emotional state. This process involves the execution of product information APIs and online search algorithms. Input consists of keywords and intent information, while output is a list of relevant information.

[0911] Step 6:

[0912] The server generates information best suited to the user based on collected data and recognized emotional states. An AI model analyzes the data and determines the most appropriate suggestions. The input is collected information and emotional data, and the output is personalized suggestions.

[0913] Step 7:

[0914] The device displays the received suggestions on the user interface. The tone and color scheme of the suggestions are adjusted according to the user's emotional state. The input is the suggested content, and the output is the visual presentation of the information.

[0915] Step 8:

[0916] The user provides feedback on the presented information. The device records this feedback and sends it to the server. This data is used to improve the accuracy of future suggestions. The input is the user's comments and evaluations, and the output is the recorded feedback data.

[0917] Step 9:

[0918] The server analyzes the feedback and updates the AI ​​model. This process continuously improves sentiment recognition and suggestion accuracy. The input is feedback data, and the output is the updated AI model.

[0919] (Application Example 2)

[0920] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0921] Conventional information delivery systems have difficulty providing personalized suggestions that take into account the user's emotional state, resulting in challenges in improving user satisfaction. Furthermore, because the suggested content is not optimized for the user's emotional state at the time, it has been difficult to improve the user experience.

[0922] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0923] In this invention, the server includes means for analyzing user input using natural language processing to simultaneously identify the user's intent and emotional state, means for acquiring audio and video data to recognize the user's emotional state, and means for automatically collecting relevant information from the internet and analyzing it using artificial intelligence technology. This makes it possible to provide personalized suggestions that are optimal for the user's emotional state.

[0924] "User input" refers to the expression of requests or objectives made by a user through a device, either via voice or text.

[0925] "Natural language processing" is the technology that enables computers to understand, analyze, and process human language.

[0926] "Intention" refers to the purpose or wish that the user is trying to convey through their input.

[0927] "Emotional state" refers to information that indicates the emotions a user is feeling at a particular point in time.

[0928] "Audio data" refers to audio information that records the user's voice.

[0929] "Video data" refers to video information that records the user's facial expressions and movements.

[0930] "Automatically collecting relevant information from the internet" refers to the process of automatically retrieving information that is publicly available online.

[0931] "Artificial intelligence technology" is a technology that allows computers to mimic some aspects of human intelligence.

[0932] A "user interface" refers to the screens and methods of operation that a user uses when interacting with a system.

[0933] "Feedback" refers to the opinions and responses that users provide in response to suggestions they receive.

[0934] "Proposal accuracy" is an indicator of how well the proposals are tailored to the user's needs and feelings.

[0935] The system for realizing this invention uses a user terminal such as a smartphone or tablet and a server in the cloud. When the user inputs voice or text into the terminal, the terminal uses its built-in microphone and camera to collect data on the user's voice and facial expressions. This information is transmitted to the server via the internet.

[0936] On the server, natural language processing software (e.g., Google Cloud Natural Language API) is first used to analyze the user's input text and identify the user's intent. Simultaneously, an emotion recognition engine recognizes emotional states from audio and video data. This analysis utilizes speech recognition APIs (e.g., Google Cloud Speech-to-Text API) and facial expression recognition libraries (e.g., OpenCV).

[0937] Based on the user's intent and emotional state derived from this data, the server collects relevant information from the internet and analyzes it using artificial intelligence technology. The analysis utilizes machine learning models to generate personalized suggestions best suited to the user. These suggestions are sent to the user's device, and the content displayed in the user interface is adjusted in terms of color scheme and message tone to match the user's emotional state.

[0938] For example, if a user intends to relax on their day off and the server detects stress from their tone of voice, it will suggest products with relaxing effects and comfortable delivery options in a gentle tone.

[0939] An example of an input prompt for a generative AI model could be, "Please create a personalized suggestion that considers products and benefits to a user who wants to relax." Based on this prompt, the AI ​​can generate optimal suggestions that match the user's emotions.

[0940] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0941] Step 1:

[0942] Users input information into their smartphones or tablets via voice or text. The device receives this input and simultaneously captures audio and video data using its built-in microphone and camera. This collects data about the user's intent and emotional data based on voice tone and facial expressions.

[0943] Step 2:

[0944] The device converts the collected audio data into text using a speech recognition API and sends it to the server. At the same time, video data is also sent for emotion recognition and used as material to evaluate the user's emotional state from their facial expressions. This forms the input dataset.

[0945] Step 3:

[0946] The server analyzes the transcribed user input using natural language processing techniques to identify the user's specific intentions. It also uses an emotion engine to recognize emotional states based on audio and video data. The input data consists of speech recognition results and video data, while the output provides metadata of the user's intentions and emotions.

[0947] Step 4:

[0948] The server collects relevant information from the internet based on the analyzed intent and sentiment metadata. It uses AI technology to analyze product information and reviews and generate personalized suggestions for the user. In this process, the input relevant information is processed into personalized suggestions as output.

[0949] Step 5:

[0950] The generated suggestions are sent to the device to be presented with a tone and color scheme adjusted based on the user's emotional state. The device then appropriately displays the received suggestions on the user interface. At this time, the displayed content will have its color scheme and tone adjusted based on the input emotional metadata.

[0951] Step 6:

[0952] The user sends feedback on the presented suggestions from their device to the server. The server collects this feedback data as training material and uses it to improve the accuracy of the suggestions and the sentiment recognition model. This feedback loop improves the performance of the output system in subsequent uses.

[0953] For the generating AI model, the prompt "Please create a personalized suggestion that considers products and benefits to recommend when the user is feeling relaxed" is used to generate AI-based suggestions.

[0954] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0955] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0956] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0957] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0958] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0959] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0960] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0961] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0962] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0963] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0964] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0965] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0966] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0967] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0968] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0969] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0970] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0971] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0972] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0973] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0974] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0975] The following is further disclosed regarding the embodiments described above.

[0976] (Claim 1)

[0977] A means of analyzing user input using natural language processing to identify the user's intent,

[0978] A means of automatically collecting and analyzing relevant data from the internet,

[0979] A means of generating optimal suggestions for users using collected data,

[0980] A means of sending the generated proposal to the user's terminal and presenting it to the user,

[0981] A system that includes means of collecting and learning from data to improve the accuracy of suggestions based on user feedback.

[0982] (Claim 2)

[0983] The system according to claim 1, which uses the user's location information to obtain information on nearby stores and provides real-time price comparisons and optimal route suggestions.

[0984] (Claim 3)

[0985] The system according to claim 1, which generates personalized suggestions based on the user's past behavioral patterns and preferences.

[0986] "Example 1"

[0987] (Claim 1)

[0988] A means for analyzing natural language input received from a user via an information device and identifying the user's intent,

[0989] A means of automatically collecting and analyzing relevant information from a database on the internet using an information processing device,

[0990] A means of generating suggestions suitable for user requests based on collected information,

[0991] A means of transmitting the generated proposal to an information device and presenting it to the user,

[0992] A system that includes means for collecting and learning data to improve the accuracy of suggestions using user feedback information.

[0993] (Claim 2)

[0994] The system according to claim 1, which uses the user's location information to obtain information on nearby sales facilities and proposes real-time price comparisons and optimal routes.

[0995] (Claim 3)

[0996] The system according to claim 1, which generates personalized suggestions based on the user's past behavioral history and preferences.

[0997] "Application Example 1"

[0998] (Claim 1)

[0999] A means of analyzing user input using natural language processing to identify the user's intent,

[1000] A means of automatically collecting and analyzing relevant data from the internet,

[1001] A means of generating optimal suggestions for users using collected data,

[1002] A means of sending the generated proposal to the user's terminal and presenting it to the user,

[1003] A means of collecting and learning data to improve the accuracy of suggestions based on user feedback,

[1004] A system that includes means of providing users with the most suitable payment options and discount information in real time.

[1005] (Claim 2)

[1006] The system according to claim 1, which uses the user's location information to obtain information on nearby stores and provides real-time price comparisons and optimal route suggestions.

[1007] (Claim 3)

[1008] The system according to claim 1, which generates personalized suggestions based on the user's past behavior patterns and preferences and provides specific purchasing options.

[1009] "Example 2 of combining an emotion engine"

[1010] (Claim 1)

[1011] A means of analyzing user input using natural language processing technology to identify the user's intent,

[1012] A means of recognizing emotional states based on voice tone and facial expression data acquired during user input,

[1013] A means of automatically collecting and analyzing relevant information from the internet,

[1014] A means of generating information best suited to the user by utilizing collected information and the user's emotional state,

[1015] A means of sending the generated information to the user's terminal and presenting it with a customized interface based on the emotional state,

[1016] A system that includes means of collecting and learning from data based on user feedback to improve the accuracy of information.

[1017] (Claim 2)

[1018] The system according to claim 1, which uses the user's location information to obtain information on nearby stores and provides real-time price comparisons and optimal route suggestions.

[1019] (Claim 3)

[1020] The system according to claim 1, which generates personalized information based on the user's past behavioral patterns and preferences.

[1021] "Application example 2 when combining with an emotional engine"

[1022] (Claim 1)

[1023] A means of analyzing user input using natural language processing to simultaneously identify the user's intent and emotional state,

[1024] A means for acquiring audio and video data to recognize the user's emotional state,

[1025] A means of automatically collecting relevant information from the internet and analyzing it using artificial intelligence technology,

[1026] A means of generating personalized suggestions that are optimal for the user based on the user's emotional state and analysis results,

[1027] A means of presenting the generated suggestions on the user interface with the optimal color scheme and tone, tailored to the user's emotional state.

[1028] A system that includes means for collecting and learning data to improve suggestion accuracy and sentiment recognition accuracy based on user feedback.

[1029] (Claim 2)

[1030] The system according to claim 1, which uses the user's location information to obtain information on nearby retailers and proposes real-time price comparisons and optimal routes.

[1031] (Claim 3)

[1032] The system according to claim 1, which generates personalized suggestions based on the user's past behavioral patterns, preferences, and emotional state. [Explanation of symbols]

[1033] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of analyzing user input using natural language processing to identify the user's intent, A means of automatically collecting and analyzing relevant data from the internet, A means of generating optimal suggestions for users using collected data, A means of sending the generated proposal to the user's terminal and presenting it to the user, A means of collecting and learning data to improve the accuracy of suggestions based on user feedback, A system that includes means of providing users with the most suitable payment options and discount information in real time.

2. The system according to claim 1, which uses the user's location information to obtain information on nearby stores and provides real-time price comparisons and optimal route suggestions.

3. The system according to claim 1, which generates personalized suggestions based on the user's past behavior patterns and preferences and provides specific purchasing options.

Images