system

Abstract

Provide a system. 【Solution means】 A device for receiving input in natural language from a user, A device for analyzing the input in natural language to identify the user's intention, A device for searching for relevant products from an information aggregate based on the user's intention, A device for generating a presentation to the user from the searched products, A device for executing the purchase procedure of the product selected by the user based on the presentation, A device for receiving the profit for the purchase of the product from the provider, A device for dynamically presenting relevant information of the selected product to the user, A system including the above.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, there are numerous services on the Internet, and users need to spend a great deal of time and effort to find the most suitable service for their needs. Also, due to an excessive number of options, users often get lost in choosing which service to select and end up making choices with low satisfaction. Furthermore, on the service provider side, there is also a problem of opportunity loss because they cannot properly match with customers. To solve these problems, it is necessary to provide a system that allows users to easily find and select the most suitable service.

Means for Solving the Problems

[0005] This invention provides a means for receiving natural language input from a user, analyzing that input, and identifying the user's intent. Furthermore, it includes means for searching a database for relevant services based on the identified intent, and means for proposing the most suitable service to the user from the search results. It also includes means for easily executing the procedure for purchasing the service selected by the user, thereby enabling the user to quickly and efficiently select and use the service best suited to their needs. In addition, after the purchase procedure is completed, the system can effectively receive revenue from the service provider, providing convenience to the service provider as well.

[0006] A "user" refers to an individual or organization that operates a system and seeks out services that meet their specific needs.

[0007] "Natural language" refers to all languages ​​that humans use on a daily basis, and in particular, to languages ​​that enable natural conversational input to computers.

[0008] "Input" refers to the act of providing information to a system by a user, and can take the form of text or audio.

[0009] "Analysis" refers to the process by which a system understands user input and identifies its meaning and intent.

[0010] "Intention" refers to the goal or desire that the user is trying to achieve through their input.

[0011] "Service" refers to a product, information, or function provided to meet a user's specific needs.

[0012] A "database" refers to a recording medium in which information related to a service is systematically stored and can be searched and retrieved.

[0013] "Searching" refers to a series of processes that retrieve service information from a database that matches the user's needs.

[0014] "Proposal" refers to the provision of information to notify users of appropriate services discovered through search and to encourage their use.

[0015] "Purchase procedure" refers to the execution of procedures such as contracts and payments required for the actual use of the service selected by the user.

[0016] "Revenue" refers to the consideration paid from the service provider to the system operator and indicates the economic benefits associated with the provision or introduction of the service.

[0017] "Receipt" refers to the act of the system operator receiving the revenue from the service provider.

Brief Explanation of Drawings

[0018] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10]Shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0019] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0020] First, the language used in the following description will be explained.

[0021] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0022] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0023] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0024] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0026] [First Embodiment]

[0027] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0028] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0031] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0034] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0038] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0039] The concierge AI system according to the present invention provides a function that analyzes the user's natural language input and suggests relevant services. This system includes a series of processes to simplify user operation and facilitate the selection of the optimal service.

[0040] Users can access the system via devices such as smartphones and PCs and input their needs and preferences in natural language. This input can be in text or voice, providing a flexible interface. The device sends the user input to the server, and the analysis process begins.

[0041] The server analyzes the received natural language input using a natural language processing (NLP) module. This analysis converts the user's intent into specific keywords and phrases, helping to identify the services the user truly desires. The NLP results are recognized as categories and actions that align with the user's wishes.

[0042] Next, the server searches the database based on the analyzed information to identify services that match the user's intent. This search process takes into account factors such as the service provider, features, availability, and cost. If multiple services are identified as search results, the optimal service candidates are listed.

[0043] For example, if a user enters "I want to go on a trip this weekend," the server analyzes this and extracts the keywords "travel" and "weekend." It then searches the database for travel-related service information and generates a list of suitable travel packages and accommodations. The user can then view this list on their device screen and select the services that interest them.

[0044] When a user selects a service, the terminal sends that information back to the server, which then begins the purchase process. Once the purchase is confirmed, the transaction is completed, the server receives the revenue from the service provider, and notifies the system operator or administrator.

[0045] In this way, the concierge AI system can simplify the process for users to quickly select the appropriate service, thereby maximizing its effectiveness.

[0046] The following describes the processing flow.

[0047] Step 1:

[0048] Users input their wishes and requests in natural language using the device. Input can be done via text or voice, and the device acquires this as digital data.

[0049] Step 2:

[0050] The terminal sends the acquired user input data to the server. This transmission takes place in real time via the internet.

[0051] Step 3:

[0052] The server passes the received natural language data to a natural language processing (NLP) module. Within the server, this data is tokenized, and important keywords and phrases are extracted.

[0053] Step 4:

[0054] The server identifies relevant categories and actions based on the user's intent identified by NLP. This clarifies the information that addresses the user's needs.

[0055] Step 5:

[0056] The server searches the database for services that match the identified intent. This search prioritizes highly relevant services and also takes into account the service's characteristics and terms of service.

[0057] Step 6:

[0058] The server generates a list of the most suitable services to suggest to the user based on the search results. This list is structured to best match the user's preferences.

[0059] Step 7:

[0060] The terminal displays a list of service suggestions sent from the server to the user. The user can review this list and select the service they like.

[0061] Step 8:

[0062] When a user selects a service, the device sends that selection information back to the server. This selection information includes basic data necessary for the purchase.

[0063] Step 9:

[0064] After receiving the user's selection information, the server executes the necessary purchase procedures. This includes service reservations and payment processing.

[0065] Step 10:

[0066] The server notifies the corresponding service provider after the purchase process is complete and calculates and manages the revenue. Revenue from the service provider is recorded and later reported to the operator.

[0067] (Example 1)

[0068] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0069] Traditional information provision systems required users to spend a great deal of time and effort searching for the information and services they needed, resulting in a poor user experience during the selection and purchase process. Furthermore, the lack of technology to effectively utilize voice input and accurately grasp user intent using natural language meant that user needs could not be fully met.

[0070] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0071] In this invention, the server includes a device for receiving natural language input from a user, a processing device for analyzing the natural language input and identifying the user's intent, and a computing device for retrieving relevant information from a storage device based on the user's intent. This enables the user to quickly and accurately obtain information and services that match their needs and enjoy a comfortable selection and purchasing experience.

[0072] A "device" is a machine or electronic device designed to perform a specific function.

[0073] A "processing device" is a computer or dedicated hardware that analyzes input information and performs calculations or operations to execute a specified task.

[0074] A "storage device" is a piece of hardware or system used to store data and information, and to retrieve and use it as needed.

[0075] A "processing unit" is a mechanism or electronic circuit that performs calculations and logical decisions based on input data and outputs the results.

[0076] A "transmitting device" is a machine or program that has the function of transmitting information to another device or system.

[0077] A "receiving device" is an electronic device or system that receives, analyzes, or displays information transmitted from an external source.

[0078] A "conversion device" is a system or circuit designed to change data from one format to another.

[0079] An "analysis device" is hardware or software used to structure information and break it down or understand it according to a specific purpose.

[0080] A "management device" refers to equipment or a program that has the function of monitoring and controlling a system or process.

[0081] This invention is a system that quickly suggests relevant information and services based on information entered by the user in natural language. The system consists of a server, a terminal, and a user, and aims to improve the overall user experience.

[0082] User actions

[0083] Users access the system using devices such as smartphones and PCs. Here, they can input their needs and preferences using natural language, i.e., text or voice. If voice input is selected, the device has a built-in conversion device that converts speech to text. This function utilizes speech recognition technology, enabling efficient text conversion of voice input.

[0084] Terminal processing

[0085] The terminal sends user input to the server. Encrypted communication is often used for data transmission to ensure security and speed. This transmission allows the server to receive the user's request.

[0086] Server-based processing

[0087] The server uses an NLP (Natural Language Processing) module to analyze user input. This module converts the input into important keywords and phrases to understand the user's intent. Common APIs and libraries may be used for this analysis.

[0088] Based on the analysis results, the server searches for relevant information stored in its memory. A database management system is used to efficiently find services that meet the user's needs. For example, if a user enters "I want to go on a trip this weekend," the server analyzes this and extracts keywords such as "travel" and "weekend."

[0089] Service proposal and purchase procedure

[0090] The server sends information to the user's terminal to suggest multiple options obtained through the search. The user can review these suggestions on the screen and select those that interest them. An example of a suggestion message would be: "The user is looking for a weekend trip. Please suggest recommended travel packages."

[0091] Once the user makes a selection and decides to purchase the service, the device sends that information back to the server. The server initiates the purchase process and uses the payment system to ensure the transaction is completed securely. Once payment is complete, the system receives the payment from the supplier and notifies the administrator of this information, thus completing the entire process.

[0092] In this way, the system provides users with information quickly and efficiently, and also provides an environment that allows for a smooth purchasing process.

[0093] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0094] Step 1:

[0095] The user uses the device to input natural language. For example, the user might input text or voice, such as "I want to go on a trip this weekend." The device receives this user input and converts the voice to text if necessary. The resulting input data is natural language text.

[0096] Step 2:

[0097] The terminal sends the obtained text to the server. A secure communication protocol is used for transmission. The server receives natural language text as input data from the terminal's output.

[0098] Step 3:

[0099] The server processes the received natural language text through a natural language processing (NLP) module. At this stage, the server analyzes the input data and identifies the user's intent. Specifically, keywords such as "travel" and "weekend" are extracted. This results in the intent analysis results being output.

[0100] Step 4:

[0101] The server searches its internal database based on the intent analysis results. The server retrieves information related to the keywords from its storage and finds services that match the user's needs. The output is a list of search results.

[0102] Step 5:

[0103] The server sends a list of search results to the device. The list includes multiple services and products that match the user's preferences. The device converts the received data into a UI / UX format for display to the user.

[0104] Step 6:

[0105] The user selects a service of interest from a list of options displayed on the device screen. Once a selection is made, the device sends this selection information to the server. This is where data about the user's selected service is input.

[0106] Step 7:

[0107] The server initiates the purchase process based on the user's selection, integrating with an external payment system to process the transaction. Once payment is confirmed, the server acknowledges the transaction's completion and notifies both the user and the service provider that the transaction was successful. A purchase confirmation notification is generated as output.

[0108] (Application Example 1)

[0109] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0110] In modern e-commerce, it is not easy for users to find the best products for themselves in a short amount of time. Furthermore, there is a need for efficient methods to provide appropriate suggestions from a vast amount of product information and to facilitate the purchase process smoothly. Traditional systems often have a gap between user demand and the products offered, which can easily diminish purchasing intent.

[0111] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0112] In this invention, the server includes a device for receiving natural language input from a user, a device for analyzing the natural language input to identify the user's intent, and a device for searching for relevant products from a set of information based on the user's intent. This makes it possible to quickly and accurately present product information that meets the user's requirements.

[0113] A "user" is someone who utilizes the system, inputting their own wishes and requests and receiving suggestions in response.

[0114] "Natural language" refers to the language system that humans use on a daily basis, and which is expressed in either spoken or written form.

[0115] "Input" refers to the act of a user providing information or data to a system.

[0116] A "device" is a mechanical or system component designed to perform a specific function or role.

[0117] "Analysis" is the process of breaking down data and information to understand their meaning and structure.

[0118] "Intention" refers to the goals and desires that a user tries to achieve based on their own hopes and objectives.

[0119] An "information collection" is a systematic database or repository that organizes and manages diverse data and information.

[0120] "Exploration" is the act of searching for data in order to find the information or results that are being sought.

[0121] A "product" is an object that is sold as a tangible item or an intangible service.

[0122] "Presentation" refers to the act of showing information or suggestions to a user and prompting them to make a choice or decision.

[0123] For this invention to be implemented, coordination between the server and the user terminal is crucial. The server receives information entered by the user in natural language and processes it to analyze its context and keywords. Natural language processing technologies such as Google® Cloud NLP API are used for the analysis to extract the input content as specific requests and intentions.

[0124] The extracted information is then passed on to a process that searches for relevant product data in a database, which is a collection of information. This database search uses a high-speed data management system, such as Firebase. The resulting product list is then presented to the user's device, which could be a smartphone or a personal computer.

[0125] Users can select their desired items from the presented products, and the server proceeds with the purchase process based on their selection. An electronic payment platform is integrated into the purchase process to support smooth transaction completion.

[0126] For example, if a user enters "What do I need for a healthy breakfast?", the server will begin analyzing the request and suggest corresponding ingredients and recipes. Furthermore, if the user enters a prompt such as "What foods are recommended for a weekend picnic?", related products such as sandwiches and snacks will be listed. Throughout this entire process, users can easily and efficiently obtain the products and services they need.

[0127] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0128] Step 1:

[0129] The user inputs their request into the device using natural language. For example, they might text or voice a request saying, "Tell me what I need for a healthy breakfast." The device receives this input and prepares to send it to the server. Regardless of the format or type of input, accurately conveying the user's intent is crucial.

[0130] Step 2:

[0131] The server receives natural language input from the user via the terminal. Next, it analyzes the input text using natural language processing technologies such as the Google Cloud NLP API. During the analysis process, keywords are extracted to understand the user's request and grasp the context. This analysis outputs relevant phrases such as "healthy breakfast" and "needs."

[0132] Step 3:

[0133] Based on the analysis results, the server searches databases such as Firebase. The previously extracted keywords are used as search queries to find relevant product information. The information retrieved from the database is output as a list of products that match the user's request.

[0134] Step 4:

[0135] The server returns the retrieved product list to the terminal. The terminal receives this list and presents it visually to the user. The list includes detailed information such as product name, description, and price, which the user can compare to increase their purchase intent.

[0136] Step 5:

[0137] Once the user selects the items they wish to purchase, the server receives this selection and initiates the purchase process. Based on the information of the selected items, it prepares to execute the transaction via the electronic payment platform. The purpose of this step is to complete the transaction securely and quickly.

[0138] Step 6:

[0139] After a transaction is completed, the server records the details of the purchase and notifies the service provider of the information. Finally, the server aggregates and manages this information to ensure transaction transparency and efficient management.

[0140] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0141] The concierge AI system incorporating the emotion engine according to the present invention analyzes the user's natural language input and further recognizes the user's emotions from the input to provide more personalized services. This system includes an advanced process for making appropriate and effective service suggestions in response to the user's input.

[0142] Users input their wishes and requests into the system via text or voice through their device. The emotion engine then analyzes the input information to identify emotional nuances. As a result, the type of emotion the user is experiencing—such as joy, sadness, surprise, or anger—becomes clear.

[0143] The terminal sends user input data to the server, which uses natural language processing (NLP) to analyze the input in detail. This extracts the user's specific intentions. Furthermore, emotional information recognized by the emotion engine is also processed by the server, and an appropriate service category is set according to the user's state.

[0144] The server searches the database based on the analysis results and emotional information to identify relevant service candidates. During this process, suggestions are made that take the user's emotional state into account, and the priority of recommended services is adjusted based on emotional data. This process might, for example, suggest services suitable for relaxation if the user is stressed, or more stimulating services if they are excited.

[0145] For example, if a user types "Tell me how to relieve stress," the server will respond by extracting keywords such as "stress relief" and "methods." The emotion engine senses the user's stress level from their voice tone and text expression, and suggests entertainment, health-related services, or events that are most appropriate for that state.

[0146] The list of suggestions sent to the terminal is displayed to the user, who can then make a selection based on it. The introduction of an emotion engine enables suggestions that harmonize with the user's intuition and state of mind, further improving the user experience. The server executes the service purchase procedure according to the user's selection and manages the final revenue. In this way, the system utilizing the emotion engine provides a more refined service experience by simultaneously considering the user's emotions and needs.

[0147] The following describes the processing flow.

[0148] Step 1:

[0149] The user enters their wishes and requests into the terminal, and input can be done via text or voice. The terminal then retrieves the input data and prepares for the next process.

[0150] Step 2:

[0151] The device sends the acquired input data to the server. At the same time, in the case of voice input, it prepares the emotion engine to start the emotion analysis process.

[0152] Step 3:

[0153] The server analyzes the received natural language data using a natural language processing (NLP) module. This analysis extracts keywords and phrases related to the user's intent.

[0154] Step 4:

[0155] The server uses an emotion engine to recognize the user's emotions from the input data. The analyzed emotion information is used to identify the user's emotional state (e.g., joy, sadness, surprise, anger).

[0156] Step 5:

[0157] The server searches the database for relevant services based on the analyzed intent and emotional information. Based on the emotional information, the service candidates that best match the user's mental state are selected preferentially.

[0158] Step 6:

[0159] The server organizes the services obtained from the search results and creates a suggested list that is tailored based on the user's feelings and intentions. This suggested list contains the services that are most suitable for the user.

[0160] Step 7:

[0161] The terminal displays a list of suggestions sent from the server to the user. The user can then select the service that best suits their needs from the suggested options.

[0162] Step 8:

[0163] When a user selects a specific service, the device sends that information back to the server. The server then initiates the purchase or reservation process.

[0164] Step 9:

[0165] The server executes the purchase process for the selected service and completes the requested actions. Finally, the server receives and records the revenue from the service provider.

[0166] (Example 2)

[0167] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0168] Conventional information provision systems only suggest services based on user input, without considering the user's emotional state. Therefore, they were unable to provide personalized services tailored to the user's emotions, resulting in insufficient improvement in user satisfaction.

[0169] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0170] In this invention, the server includes means for receiving natural language input from a user, means for analyzing the input to identify the user's intent, and means for analyzing the emotions contained in the input using an emotion engine. This enables personalized service suggestions based on the user's emotional state.

[0171] A "user" refers to a person who provides input to the system using natural language.

[0172] "Natural language" refers to the words and sentences that humans use in everyday life, and is information that should be processed in order to be understood by computers.

[0173] "Input" refers to information that a user provides to the system in the form of text or voice.

[0174] "Analysis" is the process of examining input natural language syntactically and semantically and converting the information into a format that is easy to understand.

[0175] "Intention" refers to the purpose or request that the user is trying to convey through their input.

[0176] An "emotion engine" is a technology used to identify emotional nuances from user input.

[0177] A "database" refers to a searchable storage system that organizes and stores related information and data.

[0178] A "generative AI model" is a system that uses machine learning to automatically generate responses and suggestions based on input data.

[0179] "Suggestions" refer to potentially useful information or services provided to the user as a result of analysis and database searches.

[0180] "Purchase process" refers to all operations and arrangements necessary to enable the user to use the service they have selected.

[0181] "Revenue" refers to the financial profit that the system operator receives as payment for the services provided.

[0182] In this invention, the user uses a terminal to perform natural language input. The input is sent to the server in the form of text or voice. The server utilizes a natural language processing engine to analyze the received input data and identify the user's intent. In this process, the grammar and meaning of the input are examined in detail to understand what the user's request is.

[0183] Next, the server uses an emotion engine to analyze the emotions contained in the user's input. This engine can identify nuances of emotion based on the tone of voice and text expression. As a result of the analysis, the user's emotional state is classified into categories such as joy, sadness, surprise, and anger.

[0184] The server searches the database for relevant information based on the user's intent and emotional information. This database stores a variety of services and suggestions, and searches are performed according to the user's state.

[0185] Using a generative AI model, the server automatically generates service suggestions tailored to the user based on the extracted information. These suggestions are then sent to the terminal and displayed to the user. The user can then select the most appropriate service from the displayed options.

[0186] As a concrete example, consider a scenario where a user types "Tell me how to relieve stress" into their device. In this case, the server analyzes keywords such as "stress relief" and "methods." The emotion engine detects the user's stress level from their input. Based on this, the server uses a generating AI model to suggest music, exercise, or other relaxation-related activities. An example of a prompt could be, "Please suggest the most suitable entertainment service for when the user is feeling relaxed."

[0187] This system aims to improve user satisfaction by enabling suggestions that take into account both the user's emotions and intentions.

[0188] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0189] Step 1:

[0190] Users input information in natural language through their devices. Input can be in text or voice format. The input information is sent to the server in its original format. For example, if a user says to their smartphone, "I want to know how to relax," the voice data is recorded and converted into text.

[0191] Step 2:

[0192] The terminal sends the input natural language data to the server. The input is the user's voice or text data, and the output is data packets sent to the server via the network. The data is transferred in real time, allowing the process to proceed to the next step without delay.

[0193] Step 3:

[0194] The server passes the received data to the natural language processing engine. The input is the user's text data, and the output is information about the analyzed grammatical structure and the user's intent. The natural language processing engine performs grammatical analysis on the input data and extracts keywords such as "relax" and "method."

[0195] Step 4:

[0196] The server activates an emotion engine to analyze the user's emotions from their input. The input is text data such as "I want to know how to relax," and the output is emotional information such as "The user is feeling stressed." In the case of voice input, the tone of voice is also used for emotion analysis.

[0197] Step 5:

[0198] The server searches the database based on the analyzed intent and sentiment information. The input is the analyzed information regarding intent and sentiment, and the output is information on multiple related services. The database search finds the most suitable service based on specific keywords or sentiments.

[0199] Step 6:

[0200] Using a generative AI model, the server generates suggestions for the user. The input consists of service information retrieved from a database and the user's emotional state, while the output is a specific service suggestion. The prompt used is "Suggest the most suitable entertainment service for when the user is feeling relaxed."

[0201] Step 7:

[0202] The terminal displays suggestions. The output from the server is a list of specific services displayed on the user's screen, and the input is suggestion data sent from the server. The user selects a service based on this list.

[0203] Step 8:

[0204] Based on the service selected by the user, the server executes the purchase process. The input is the user's selection data, and the output is purchase information sent to the service provider and a confirmation message to the user. Specific actions include confirming purchase information, executing payment, and confirming the service reservation.

[0205] (Application Example 2)

[0206] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0207] Traditional personalized service delivery systems can analyze user intent, but they have a challenge in that they do not adequately provide suggestions based on user emotions. Therefore, it is necessary to improve the user experience by suggesting appropriate services according to the user's emotional state.

[0208] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0209] In this invention, the server includes means for receiving natural language input from a user, means for analyzing the natural language input to identify the user's intent and emotional information, and means for searching a database for relevant services based on the user's intent and emotional information. This enables personalized service suggestions that correspond to the user's emotional state.

[0210] "Natural language input" refers to a method of inputting words that users use in their daily lives directly into the system.

[0211] "Analysis" is the process of examining input information in detail and interpreting its meaning and intent.

[0212] "User intent" refers to the requirements and desires that users have for the system.

[0213] "Emotional information" refers to data about a user's psychological state that can be gleaned from their statements and text.

[0214] A "database" is a structured collection of data used to efficiently store, retrieve, and manage specific information.

[0215] "Personalized suggestions" refers to the provision of services that are individually customized to suit each user's characteristics and circumstances.

[0216] "Priority" refers to the order in which multiple options are considered important or should be dealt with first.

[0217] The "purchase process" is a series of operations that transfer ownership of the selected service to the user.

[0218] "Revenue" refers to the monetary profit obtained from the service provider.

[0219] This invention configures a system in a specific way to process natural language input from a user. When a user inputs instructions into the terminal via speech or text, the terminal uses a speech recognition API to convert the speech into text data. Next, a natural language processing library is used to analyze the text data and extract the user's intent and sentiment information.

[0220] The server searches a large database for relevant services based on extracted intent and sentiment information. It further analyzes the emotional state using a sentiment analysis API to determine the priority of services best suited to the user's emotions. This enables more personalized recommendations.

[0221] This result can be returned to the device, displaying the most suitable service suggestions to the user based on their emotions. Based on the suggested service, the purchase process is executed, and revenue from the service provider is managed simultaneously. This revenue management ensures mutually beneficial transactions for both the user and the service provider.

[0222] As a concrete example, if a user types "Tell me how to relax" into their device, the system uses an emotion engine to identify the user's stress level. Based on this, it can suggest things like meditation apps, relaxing music, or guided yoga classes. An example of a prompt to the related generative AI model would be, "The user is seeking relaxation. Please list suggestions that can help relieve stress, such as music, aromatherapy, and yoga classes."

[0223] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0224] Step 1:

[0225] The user provides input to the device via voice or text. If voice input is received, the device uses a speech recognition API to convert it into text data. The input data may include the user's desire to relax.

[0226] Step 2:

[0227] The server analyzes the text data received from the terminal using a natural language processing library. This analysis extracts the user's intent as "seeking relaxation." The output provides keywords related to the user's intent.

[0228] Step 3:

[0229] The server uses a sentiment analysis API to analyze the sentiment information contained in the text data. Emotions (e.g., stress) are identified from words and phrases in the input data. The resulting sentiment information is then output.

[0230] Step 4:

[0231] The server searches the database based on the extracted intent and emotional information. This search process filters services and content related to "relaxation," listing the most suitable service candidates for that emotion. The output is a prioritized list of services.

[0232] Step 5:

[0233] The terminal displays a list of service options sent from the server to the user. The user can then select a service of interest from this list. This executes a selection action on the user interface.

[0234] Step 6:

[0235] The server executes the purchase process for the service selected by the user. The input is the service information selected by the user, and the output generates purchase confirmations and payment completion notifications. The server receives and manages revenue from the service provider.

[0236] Step 7:

[0237] The server sends a prompt message to the generative AI model based on the service suggestions. Specifically, a prompt message like, "The user is seeking relaxation. Please list suggestions that can help relieve stress, such as music, aromatherapy, and yoga classes," is generated. This allows the generative AI model to update its database to provide more personalized services.

[0238] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0239] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0240] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0241] [Second Embodiment]

[0242] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0243] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0244] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0245] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0246] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0247] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0248] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0249] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0250] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0251] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0252] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0253] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0254] The concierge AI system according to the present invention provides a function that analyzes the user's natural language input and suggests relevant services. This system includes a series of processes to simplify user operation and facilitate the selection of the optimal service.

[0255] Users can access the system via devices such as smartphones and PCs and input their needs and preferences in natural language. This input can be in text or voice, providing a flexible interface. The device sends the user input to the server, and the analysis process begins.

[0256] The server analyzes the received natural language input using a natural language processing (NLP) module. This analysis converts the user's intent into specific keywords and phrases, helping to identify the services the user truly desires. The NLP results are recognized as categories and actions that align with the user's wishes.

[0257] Next, the server searches the database based on the analyzed information to identify services that match the user's intent. This search process takes into account factors such as the service provider, features, availability, and cost. If multiple services are identified as search results, the optimal service candidates are listed.

[0258] For example, if a user enters "I want to go on a trip this weekend," the server analyzes this and extracts the keywords "travel" and "weekend." It then searches the database for travel-related service information and generates a list of suitable travel packages and accommodations. The user can then view this list on their device screen and select the services that interest them.

[0259] When a user selects a service, the terminal sends that information back to the server, which then begins the purchase process. Once the purchase is confirmed, the transaction is completed, the server receives the revenue from the service provider, and notifies the system operator or administrator.

[0260] In this way, the concierge AI system can simplify the process for users to quickly select the appropriate service, thereby maximizing its effectiveness.

[0261] The following describes the processing flow.

[0262] Step 1:

[0263] Users input their wishes and requests in natural language using the device. Input can be done via text or voice, and the device acquires this as digital data.

[0264] Step 2:

[0265] The terminal sends the acquired user input data to the server. This transmission takes place in real time via the internet.

[0266] Step 3:

[0267] The server passes the received natural language data to a natural language processing (NLP) module. Within the server, this data is tokenized, and important keywords and phrases are extracted.

[0268] Step 4:

[0269] The server identifies relevant categories and actions based on the user's intent identified by NLP. This clarifies the information that addresses the user's needs.

[0270] Step 5:

[0271] The server searches the database for services that match the identified intent. This search prioritizes highly relevant services and also takes into account the service's characteristics and terms of service.

[0272] Step 6:

[0273] The server generates a list of the most suitable services to suggest to the user based on the search results. This list is structured to best match the user's preferences.

[0274] Step 7:

[0275] The terminal displays a list of service suggestions sent from the server to the user. The user can review this list and select the service they like.

[0276] Step 8:

[0277] When the user selects a service, the terminal returns the selection information to the server. The selection information also includes the basic data required for purchase.

[0278] Step 9:

[0279] After receiving the selection information from the user, the server executes the necessary purchase procedures. This includes service reservation and payment processing.

[0280] Step 10:

[0281] After the completion of the purchase procedures, the server notifies the corresponding service provider and calculates and manages the revenue. The revenue from the service provider is recorded and later reported to the operator.

[0282] (Example 1)

[0283] Next, Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0284] In the conventional information providing system, it takes a great deal of time and effort for the user to search for the information and services they need, and there is a problem that the user experience deteriorates during the selection and purchase procedures. In addition, since there is a lack of technology to effectively utilize voice input and accurately grasp the user's intention using natural language, there is a problem that the user's needs cannot be fully satisfied.

[0285] (Example 1) The specific processing by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0286] In this invention, the server includes a device that receives input in natural language from a user, a processing device that analyzes the input in natural language to identify the user's intention, and an arithmetic device that searches for relevant information from a storage device based on the user's intention. As a result, the user can quickly and accurately obtain information and services that meet their needs, and enjoy a comfortable selection and purchase experience.

[0287] A "device" refers to a machine or electronic equipment designed to perform a specific function.

[0288] A "processing device" refers to a computer or dedicated hardware that analyzes the input information and performs calculations or operations to execute a specified task.

[0289] A "storage device" refers to hardware or a system for storing data and information so that it can be retrieved and used as needed.

[0290] An "arithmetic device" refers to a mechanism or electronic circuit that performs calculations and logical judgments based on the input data and outputs the results.

[0291] A "transmitting device" refers to a machine or program having a function for transmitting information to other devices or systems.

[0292] A "receiving device" refers to an electronic device or system for receiving information transmitted from the outside and analyzing or displaying it.

[0293] A "conversion device" refers to a system or circuit designed to convert data in one format into another format.

[0294] An "analyzing device" refers to hardware or software for structuring information and decomposing or understanding it according to a specific purpose.

[0295] A "management device" refers to equipment or a program that has the function of monitoring and controlling a system or process.

[0296] This invention is a system that quickly suggests relevant information and services based on information entered by the user in natural language. The system consists of a server, a terminal, and a user, and aims to improve the overall user experience.

[0297] User actions

[0298] Users access the system using devices such as smartphones and PCs. Here, they can input their needs and preferences using natural language, i.e., text or voice. If voice input is selected, the device has a built-in conversion device that converts speech to text. This function utilizes speech recognition technology, enabling efficient text conversion of voice input.

[0299] Terminal processing

[0300] The terminal sends user input to the server. Encrypted communication is often used for data transmission to ensure security and speed. This transmission allows the server to receive the user's request.

[0301] Server-based processing

[0302] The server uses an NLP (Natural Language Processing) module to analyze user input. This module converts the input into important keywords and phrases to understand the user's intent. Common APIs and libraries may be used for this analysis.

[0303] Based on the analysis result, the server searches for relevant information stored in the storage device. To search for information, a database management system can be used to efficiently find services that meet the user's needs. As a specific example, when the user inputs "want to travel on weekends", the server analyzes this and extracts keywords such as "travel" and "weekends".

[0304] Service Proposal and Purchase Procedure

[0305] The server sends the information to the terminal to propose multiple options obtained through the search to the user. The user can view these proposals on the screen and select the ones they are interested in. A specific example of a proposal text is "The user hopes to travel on weekends. Please propose recommended travel packages."

[0306] When the user makes a selection and decides to purchase the service, the terminal sends that information back to the server. The server starts the purchase procedure and uses the payment system to ensure that the transaction is completed safely. When the payment is completed, it receives the consideration from the provider and notifies the administrator of that information, thus completing the process of the entire system.

[0307] In this way, the system provides information to the user quickly and efficiently, and further provides an environment where the purchase procedure can be carried out smoothly.

[0308] The flow of the specific process in Example 1 will be described using FIG. 11.

[0309] Step 1:

[0310] The user uses the terminal to make an input in natural language. What the user inputs is, for example, the text or voice of "want to travel on weekends". The terminal receives this user input and converts the voice to text if text conversion is required. As the input data, natural language text is obtained.

[0311] Step 2:

[0312] The terminal sends the obtained text to the server. A secure communication protocol is used for transmission. The server receives natural language text as input data from the terminal's output.

[0313] Step 3:

[0314] The server processes the received natural language text through a natural language processing (NLP) module. At this stage, the server analyzes the input data and identifies the user's intent. Specifically, keywords such as "travel" and "weekend" are extracted. This results in the intent analysis results being output.

[0315] Step 4:

[0316] The server searches its internal database based on the intent analysis results. The server retrieves information related to the keywords from its storage and finds services that match the user's needs. The output is a list of search results.

[0317] Step 5:

[0318] The server sends a list of search results to the device. The list includes multiple services and products that match the user's preferences. The device converts the received data into a UI / UX format for display to the user.

[0319] Step 6:

[0320] The user selects a service of interest from a list of options displayed on the device screen. Once a selection is made, the device sends this selection information to the server. This is where data about the user's selected service is input.

[0321] Step 7:

[0322] The server initiates the purchase process based on the user's selection, integrating with an external payment system to process the transaction. Once payment is confirmed, the server acknowledges the transaction's completion and notifies both the user and the service provider that the transaction was successful. A purchase confirmation notification is generated as output.

[0323] (Application Example 1)

[0324] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0325] In modern e-commerce, it is not easy for users to find the best products for themselves in a short amount of time. Furthermore, there is a need for efficient methods to provide appropriate suggestions from a vast amount of product information and to facilitate the purchase process smoothly. Traditional systems often have a gap between user demand and the products offered, which can easily diminish purchasing intent.

[0326] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0327] In this invention, the server includes a device for receiving natural language input from a user, a device for analyzing the natural language input to identify the user's intent, and a device for searching for relevant products from a set of information based on the user's intent. This makes it possible to quickly and accurately present product information that meets the user's requirements.

[0328] A "user" is someone who utilizes the system, inputting their own wishes and requests and receiving suggestions in response.

[0329] "Natural language" refers to the language system that humans use on a daily basis, and which is expressed in either spoken or written form.

[0330] "Input" refers to the act of a user providing information or data to a system.

[0331] A "device" is a mechanical or system component designed to perform a specific function or role.

[0332] "Analysis" is the process of breaking down data and information to understand their meaning and structure.

[0333] "Intention" refers to the goals and desires that a user tries to achieve based on their own hopes and objectives.

[0334] An "information collection" is a systematic database or repository that organizes and manages diverse data and information.

[0335] "Exploration" is the act of searching for data in order to find the information or results that are being sought.

[0336] A "product" is an object that is sold as a tangible item or an intangible service.

[0337] "Presentation" refers to the act of showing information or suggestions to a user and prompting them to make a choice or decision.

[0338] For this invention to be implemented, coordination between the server and the user terminal is crucial. The server receives information entered by the user in natural language and processes it to analyze its context and keywords. Natural language processing technologies such as the Google Cloud NLP API are used for the analysis to extract the input content as specific requests and intentions.

[0339] The extracted information is then passed on to a process that searches for relevant product data in a database, which is a collection of information. This database search uses a high-speed data management system, such as Firebase. The resulting product list is then presented to the user's device, which could be a smartphone or a personal computer.

[0340] Users can select their desired items from the presented products, and the server proceeds with the purchase process based on their selection. An electronic payment platform is integrated into the purchase process to support smooth transaction completion.

[0341] For example, if a user enters "What do I need for a healthy breakfast?", the server will begin analyzing the request and suggest corresponding ingredients and recipes. Furthermore, if the user enters a prompt such as "What foods are recommended for a weekend picnic?", related products such as sandwiches and snacks will be listed. Throughout this entire process, users can easily and efficiently obtain the products and services they need.

[0342] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0343] Step 1:

[0344] The user inputs their request into the device using natural language. For example, they might text or voice a request saying, "Tell me what I need for a healthy breakfast." The device receives this input and prepares to send it to the server. Regardless of the format or type of input, accurately conveying the user's intent is crucial.

[0345] Step 2:

[0346] The server receives natural language input from the user via the terminal. Next, it analyzes the input text using natural language processing technologies such as the Google Cloud NLP API. During the analysis process, keywords are extracted to understand the user's request and grasp the context. This analysis outputs relevant phrases such as "healthy breakfast" and "needs."

[0347] Step 3:

[0348] Based on the analysis results, the server searches databases such as Firebase. The previously extracted keywords are used as search queries to find relevant product information. The information retrieved from the database is output as a list of products that match the user's request.

[0349] Step 4:

[0350] The server returns the retrieved product list to the terminal. The terminal receives this list and presents it visually to the user. The list includes detailed information such as product name, description, and price, which the user can compare to increase their purchase intent.

[0351] Step 5:

[0352] Once the user selects the items they wish to purchase, the server receives this selection and initiates the purchase process. Based on the information of the selected items, it prepares to execute the transaction via the electronic payment platform. The purpose of this step is to complete the transaction securely and quickly.

[0353] Step 6:

[0354] After a transaction is completed, the server records the details of the purchase and notifies the service provider of the information. Finally, the server aggregates and manages this information to ensure transaction transparency and efficient management.

[0355] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0356] The concierge AI system incorporating the emotion engine according to the present invention analyzes the user's natural language input and further recognizes the user's emotions from the input to provide more personalized services. This system includes an advanced process for making appropriate and effective service suggestions in response to the user's input.

[0357] Users input their wishes and requests into the system via text or voice through their device. The emotion engine then analyzes the input information to identify emotional nuances. As a result, the type of emotion the user is experiencing—such as joy, sadness, surprise, or anger—becomes clear.

[0358] The terminal sends user input data to the server, which uses natural language processing (NLP) to analyze the input in detail. This extracts the user's specific intentions. Furthermore, emotional information recognized by the emotion engine is also processed by the server, and an appropriate service category is set according to the user's state.

[0359] The server searches the database based on the analysis results and emotional information to identify relevant service candidates. During this process, suggestions are made that take the user's emotional state into account, and the priority of recommended services is adjusted based on emotional data. This process might, for example, suggest services suitable for relaxation if the user is stressed, or more stimulating services if they are excited.

[0360] For example, if a user types "Tell me how to relieve stress," the server will respond by extracting keywords such as "stress relief" and "methods." The emotion engine senses the user's stress level from their voice tone and text expression, and suggests entertainment, health-related services, or events that are most appropriate for that state.

[0361] The list of suggestions sent to the terminal is displayed to the user, who can then make a selection based on it. The introduction of an emotion engine enables suggestions that harmonize with the user's intuition and state of mind, further improving the user experience. The server executes the service purchase procedure according to the user's selection and manages the final revenue. In this way, the system utilizing the emotion engine provides a more refined service experience by simultaneously considering the user's emotions and needs.

[0362] The following describes the processing flow.

[0363] Step 1:

[0364] The user enters their wishes and requests into the terminal, and input can be done via text or voice. The terminal then retrieves the input data and prepares for the next process.

[0365] Step 2:

[0366] The device sends the acquired input data to the server. At the same time, in the case of voice input, it prepares the emotion engine to start the emotion analysis process.

[0367] Step 3:

[0368] The server analyzes the received natural language data using a natural language processing (NLP) module. This analysis extracts keywords and phrases related to the user's intent.

[0369] Step 4:

[0370] The server uses an emotion engine to recognize the user's emotions from the input data. The analyzed emotion information is used to identify the user's emotional state (e.g., joy, sadness, surprise, anger).

[0371] Step 5:

[0372] The server searches the database for relevant services based on the analyzed intent and emotional information. Based on the emotional information, the service candidates that best match the user's mental state are selected preferentially.

[0373] Step 6:

[0374] The server organizes the services obtained from the search results and creates a suggested list that is tailored based on the user's feelings and intentions. This suggested list contains the services that are most suitable for the user.

[0375] Step 7:

[0376] The terminal displays a list of suggestions sent from the server to the user. The user can then select the service that best suits their needs from the suggested options.

[0377] Step 8:

[0378] When a user selects a specific service, the device sends that information back to the server. The server then initiates the purchase or reservation process.

[0379] Step 9:

[0380] The server executes the purchase process for the selected service and completes the requested actions. Finally, the server receives and records the revenue from the service provider.

[0381] (Example 2)

[0382] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0383] Conventional information provision systems only suggest services based on user input, without considering the user's emotional state. Therefore, they were unable to provide personalized services tailored to the user's emotions, resulting in insufficient improvement in user satisfaction.

[0384] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0385] In this invention, the server includes means for receiving natural language input from a user, means for analyzing the input to identify the user's intent, and means for analyzing the emotions contained in the input using an emotion engine. This enables personalized service suggestions based on the user's emotional state.

[0386] A "user" refers to a person who provides input to the system using natural language.

[0387] "Natural language" refers to the words and sentences that humans use in everyday life, and is information that should be processed in order to be understood by computers.

[0388] "Input" refers to information that a user provides to the system in the form of text or voice.

[0389] "Analysis" is the process of examining input natural language syntactically and semantically and converting the information into a format that is easy to understand.

[0390] "Intention" refers to the purpose or request that the user is trying to convey through their input.

[0391] An "emotion engine" is a technology used to identify emotional nuances from user input.

[0392] A "database" refers to a searchable storage system that organizes and stores related information and data.

[0393] A "generative AI model" is a system that uses machine learning to automatically generate responses and suggestions based on input data.

[0394] "Suggestions" refer to potentially useful information or services provided to the user as a result of analysis and database searches.

[0395] "Purchase process" refers to all operations and arrangements necessary to enable the user to use the service they have selected.

[0396] "Revenue" refers to the financial profit that the system operator receives as payment for the services provided.

[0397] In this invention, the user uses a terminal to perform natural language input. The input is sent to the server in the form of text or voice. The server utilizes a natural language processing engine to analyze the received input data and identify the user's intent. In this process, the grammar and meaning of the input are examined in detail to understand what the user's request is.

[0398] Next, the server uses an emotion engine to analyze the emotions contained in the user's input. This engine can identify nuances of emotion based on the tone of voice and text expression. As a result of the analysis, the user's emotional state is classified into categories such as joy, sadness, surprise, and anger.

[0399] The server searches the database for relevant information based on the user's intent and emotional information. This database stores a variety of services and suggestions, and searches are performed according to the user's state.

[0400] Using a generative AI model, the server automatically generates service suggestions tailored to the user based on the extracted information. These suggestions are then sent to the terminal and displayed to the user. The user can then select the most appropriate service from the displayed options.

[0401] As a concrete example, consider a scenario where a user types "Tell me how to relieve stress" into their device. In this case, the server analyzes keywords such as "stress relief" and "methods." The emotion engine detects the user's stress level from their input. Based on this, the server uses a generating AI model to suggest music, exercise, or other relaxation-related activities. An example of a prompt could be, "Please suggest the most suitable entertainment service for when the user is feeling relaxed."

[0402] This system aims to improve user satisfaction by enabling suggestions that take into account both the user's emotions and intentions.

[0403] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0404] Step 1:

[0405] Users input information in natural language through their devices. Input can be in text or voice format. The input information is sent to the server in its original format. For example, if a user says to their smartphone, "I want to know how to relax," the voice data is recorded and converted into text.

[0406] Step 2:

[0407] The terminal sends the input natural language data to the server. The input is the user's voice or text data, and the output is data packets sent to the server via the network. The data is transferred in real time, allowing the process to proceed to the next step without delay.

[0408] Step 3:

[0409] The server passes the received data to the natural language processing engine. The input is the user's text data, and the output is information about the analyzed grammatical structure and the user's intent. The natural language processing engine performs grammatical analysis on the input data and extracts keywords such as "relax" and "method."

[0410] Step 4:

[0411] The server activates an emotion engine to analyze the user's emotions from their input. The input is text data such as "I want to know how to relax," and the output is emotional information such as "The user is feeling stressed." In the case of voice input, the tone of voice is also used for emotion analysis.

[0412] Step 5:

[0413] The server searches the database based on the analyzed intent and sentiment information. The input is the analyzed information regarding intent and sentiment, and the output is information on multiple related services. The database search finds the most suitable service based on specific keywords or sentiments.

[0414] Step 6:

[0415] Using a generative AI model, the server generates suggestions for the user. The input consists of service information retrieved from a database and the user's emotional state, while the output is a specific service suggestion. The prompt used is "Suggest the most suitable entertainment service for when the user is feeling relaxed."

[0416] Step 7:

[0417] The terminal displays suggestions. The output from the server is a list of specific services displayed on the user's screen, and the input is suggestion data sent from the server. The user selects a service based on this list.

[0418] Step 8:

[0419] Based on the service selected by the user, the server executes the purchase process. The input is the user's selection data, and the output is purchase information sent to the service provider and a confirmation message to the user. Specific actions include confirming purchase information, executing payment, and confirming the service reservation.

[0420] (Application Example 2)

[0421] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0422] Traditional personalized service delivery systems can analyze user intent, but they have a challenge in that they do not adequately provide suggestions based on user emotions. Therefore, it is necessary to improve the user experience by suggesting appropriate services according to the user's emotional state.

[0423] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0424] In this invention, the server includes means for receiving natural language input from a user, means for analyzing the natural language input to identify the user's intent and emotional information, and means for searching a database for relevant services based on the user's intent and emotional information. This enables personalized service suggestions that correspond to the user's emotional state.

[0425] "Natural language input" refers to a method of inputting words that users use in their daily lives directly into the system.

[0426] "Analysis" is the process of examining input information in detail and interpreting its meaning and intent.

[0427] "User intent" refers to the requirements and desires that users have for the system.

[0428] "Emotional information" refers to data about a user's psychological state that can be gleaned from their statements and text.

[0429] A "database" is a structured collection of data used to efficiently store, retrieve, and manage specific information.

[0430] "Personalized suggestions" refers to the provision of services that are individually customized to suit each user's characteristics and circumstances.

[0431] "Priority" refers to the order in which multiple options are considered important or should be dealt with first.

[0432] The "purchase process" is a series of operations that transfer ownership of the selected service to the user.

[0433] "Revenue" refers to the monetary profit obtained from the service provider.

[0434] This invention configures a system in a specific way to process natural language input from a user. When a user inputs instructions into the terminal via speech or text, the terminal uses a speech recognition API to convert the speech into text data. Next, a natural language processing library is used to analyze the text data and extract the user's intent and sentiment information.

[0435] The server searches a large database for relevant services based on extracted intent and sentiment information. It further analyzes the emotional state using a sentiment analysis API to determine the priority of services best suited to the user's emotions. This enables more personalized recommendations.

[0436] This result can be returned to the device, displaying the most suitable service suggestions to the user based on their emotions. Based on the suggested service, the purchase process is executed, and revenue from the service provider is managed simultaneously. This revenue management ensures mutually beneficial transactions for both the user and the service provider.

[0437] As a concrete example, if a user types "Tell me how to relax" into their device, the system uses an emotion engine to identify the user's stress level. Based on this, it can suggest things like meditation apps, relaxing music, or guided yoga classes. An example of a prompt to the related generative AI model would be, "The user is seeking relaxation. Please list suggestions that can help relieve stress, such as music, aromatherapy, and yoga classes."

[0438] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0439] Step 1:

[0440] The user provides input to the device via voice or text. If voice input is received, the device uses a speech recognition API to convert it into text data. The input data may include the user's desire to relax.

[0441] Step 2:

[0442] The server analyzes the text data received from the terminal using a natural language processing library. This analysis extracts the user's intent as "seeking relaxation." The output provides keywords related to the user's intent.

[0443] Step 3:

[0444] The server uses a sentiment analysis API to analyze the sentiment information contained in the text data. Emotions (e.g., stress) are identified from words and phrases in the input data. The resulting sentiment information is then output.

[0445] Step 4:

[0446] The server searches the database based on the extracted intent and emotional information. This search process filters services and content related to "relaxation," listing the most suitable service candidates for that emotion. The output is a prioritized list of services.

[0447] Step 5:

[0448] The terminal displays a list of service options sent from the server to the user. The user can then select a service of interest from this list. This executes a selection action on the user interface.

[0449] Step 6:

[0450] The server executes the purchase process for the service selected by the user. The input is the service information selected by the user, and the output generates purchase confirmations and payment completion notifications. The server receives and manages revenue from the service provider.

[0451] Step 7:

[0452] The server sends a prompt message to the generative AI model based on the service suggestions. Specifically, a prompt message like, "The user is seeking relaxation. Please list suggestions that can help relieve stress, such as music, aromatherapy, and yoga classes," is generated. This allows the generative AI model to update its database to provide more personalized services.

[0453] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0454] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0455] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0456] [Third Embodiment]

[0457] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0458] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0459] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0460] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0461] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0462] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0463] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0464] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0465] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0466] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0467] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0468] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0469] The concierge AI system according to the present invention provides a function that analyzes the user's natural language input and suggests relevant services. This system includes a series of processes to simplify user operation and facilitate the selection of the optimal service.

[0470] Users can access the system via devices such as smartphones and PCs and input their needs and preferences in natural language. This input can be in text or voice, providing a flexible interface. The device sends the user input to the server, and the analysis process begins.

[0471] The server analyzes the received natural language input using a natural language processing (NLP) module. This analysis converts the user's intent into specific keywords and phrases, helping to identify the services the user truly desires. The NLP results are recognized as categories and actions that align with the user's wishes.

[0472] Next, the server searches the database based on the analyzed information to identify services that match the user's intent. This search process takes into account factors such as the service provider, features, availability, and cost. If multiple services are identified as search results, the optimal service candidates are listed.

[0473] For example, if a user enters "I want to go on a trip this weekend," the server analyzes this and extracts the keywords "travel" and "weekend." It then searches the database for travel-related service information and generates a list of suitable travel packages and accommodations. The user can then view this list on their device screen and select the services that interest them.

[0474] When a user selects a service, the terminal sends that information back to the server, which then begins the purchase process. Once the purchase is confirmed, the transaction is completed, the server receives the revenue from the service provider, and notifies the system operator or administrator.

[0475] In this way, the concierge AI system can simplify the process for users to quickly select the appropriate service, thereby maximizing its effectiveness.

[0476] The following describes the processing flow.

[0477] Step 1:

[0478] Users input their wishes and requests in natural language using the device. Input can be done via text or voice, and the device acquires this as digital data.

[0479] Step 2:

[0480] The terminal sends the acquired user input data to the server. This transmission takes place in real time via the internet.

[0481] Step 3:

[0482] The server passes the received natural language data to a natural language processing (NLP) module. Within the server, this data is tokenized, and important keywords and phrases are extracted.

[0483] Step 4:

[0484] The server identifies relevant categories and actions based on the user's intent identified by NLP. This clarifies the information that addresses the user's needs.

[0485] Step 5:

[0486] The server searches the database for services that match the identified intent. This search prioritizes highly relevant services and also takes into account the service's characteristics and terms of service.

[0487] Step 6:

[0488] The server generates a list of the most suitable services to suggest to the user based on the search results. This list is structured to best match the user's preferences.

[0489] Step 7:

[0490] The terminal displays a list of service suggestions sent from the server to the user. The user can review this list and select the service they like.

[0491] Step 8:

[0492] When a user selects a service, the device sends that selection information back to the server. This selection information includes basic data necessary for the purchase.

[0493] Step 9:

[0494] After receiving the user's selection information, the server executes the necessary purchase procedures. This includes service reservations and payment processing.

[0495] Step 10:

[0496] The server notifies the corresponding service provider after the purchase process is complete and calculates and manages the revenue. Revenue from the service provider is recorded and later reported to the operator.

[0497] (Example 1)

[0498] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0499] Traditional information provision systems required users to spend a great deal of time and effort searching for the information and services they needed, resulting in a poor user experience during the selection and purchase process. Furthermore, the lack of technology to effectively utilize voice input and accurately grasp user intent using natural language meant that user needs could not be fully met.

[0500] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0501] In this invention, the server includes a device for receiving natural language input from a user, a processing device for analyzing the natural language input and identifying the user's intent, and a computing device for retrieving relevant information from a storage device based on the user's intent. This enables the user to quickly and accurately obtain information and services that match their needs and enjoy a comfortable selection and purchasing experience.

[0502] A "device" is a machine or electronic device designed to perform a specific function.

[0503] A "processing device" is a computer or dedicated hardware that analyzes input information and performs calculations or operations to execute a specified task.

[0504] A "storage device" is a piece of hardware or system used to store data and information, and to retrieve and use it as needed.

[0505] A "processing unit" is a mechanism or electronic circuit that performs calculations and logical decisions based on input data and outputs the results.

[0506] A "transmitting device" is a machine or program that has the function of transmitting information to another device or system.

[0507] A "receiving device" is an electronic device or system that receives, analyzes, or displays information transmitted from an external source.

[0508] A "conversion device" is a system or circuit designed to change data from one format to another.

[0509] An "analysis device" is hardware or software used to structure information and break it down or understand it according to a specific purpose.

[0510] A "management device" refers to equipment or a program that has the function of monitoring and controlling a system or process.

[0511] This invention is a system that quickly suggests relevant information and services based on information entered by the user in natural language. The system consists of a server, a terminal, and a user, and aims to improve the overall user experience.

[0512] User actions

[0513] Users access the system using devices such as smartphones and PCs. Here, they can input their needs and preferences using natural language, i.e., text or voice. If voice input is selected, the device has a built-in conversion device that converts speech to text. This function utilizes speech recognition technology, enabling efficient text conversion of voice input.

[0514] Terminal processing

[0515] The terminal sends user input to the server. Encrypted communication is often used for data transmission to ensure security and speed. This transmission allows the server to receive the user's request.

[0516] Server-based processing

[0517] The server uses an NLP (Natural Language Processing) module to analyze user input. This module converts the input into important keywords and phrases to understand the user's intent. Common APIs and libraries may be used for this analysis.

[0518] Based on the analysis results, the server searches for relevant information stored in its memory. A database management system is used to efficiently find services that meet the user's needs. For example, if a user enters "I want to go on a trip this weekend," the server analyzes this and extracts keywords such as "travel" and "weekend."

[0519] Service proposal and purchase procedure

[0520] The server sends information to the user's terminal to suggest multiple options obtained through the search. The user can review these suggestions on the screen and select those that interest them. An example of a suggestion message would be: "The user is looking for a weekend trip. Please suggest recommended travel packages."

[0521] Once the user makes a selection and decides to purchase the service, the device sends that information back to the server. The server initiates the purchase process and uses the payment system to ensure the transaction is completed securely. Once payment is complete, the system receives the payment from the supplier and notifies the administrator of this information, thus completing the entire process.

[0522] In this way, the system provides users with information quickly and efficiently, and also provides an environment that allows for a smooth purchasing process.

[0523] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0524] Step 1:

[0525] The user uses the device to input natural language. For example, the user might input text or voice, such as "I want to go on a trip this weekend." The device receives this user input and converts the voice to text if necessary. The resulting input data is natural language text.

[0526] Step 2:

[0527] The terminal sends the obtained text to the server. A secure communication protocol is used for transmission. The server receives natural language text as input data from the terminal's output.

[0528] Step 3:

[0529] The server processes the received natural language text through a natural language processing (NLP) module. At this stage, the server analyzes the input data and identifies the user's intent. Specifically, keywords such as "travel" and "weekend" are extracted. This results in the intent analysis results being output.

[0530] Step 4:

[0531] The server searches its internal database based on the intent analysis results. The server retrieves information related to the keywords from its storage and finds services that match the user's needs. The output is a list of search results.

[0532] Step 5:

[0533] The server sends a list of search results to the device. The list includes multiple services and products that match the user's preferences. The device converts the received data into a UI / UX format for display to the user.

[0534] Step 6:

[0535] The user selects a service of interest from a list of options displayed on the device screen. Once a selection is made, the device sends this selection information to the server. This is where data about the user's selected service is input.

[0536] Step 7:

[0537] The server initiates the purchase process based on the user's selection, integrating with an external payment system to process the transaction. Once payment is confirmed, the server acknowledges the transaction's completion and notifies both the user and the service provider that the transaction was successful. A purchase confirmation notification is generated as output.

[0538] (Application Example 1)

[0539] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0540] In modern e-commerce, it is not easy for users to find the best products for themselves in a short amount of time. Furthermore, there is a need for efficient methods to provide appropriate suggestions from a vast amount of product information and to facilitate the purchase process smoothly. Traditional systems often have a gap between user demand and the products offered, which can easily diminish purchasing intent.

[0541] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0542] In this invention, the server includes a device for receiving natural language input from a user, a device for analyzing the natural language input to identify the user's intent, and a device for searching for relevant products from a set of information based on the user's intent. This makes it possible to quickly and accurately present product information that meets the user's requirements.

[0543] A "user" is someone who utilizes the system, inputting their own wishes and requests and receiving suggestions in response.

[0544] "Natural language" refers to the language system that humans use on a daily basis, and which is expressed in either spoken or written form.

[0545] "Input" refers to the act of a user providing information or data to a system.

[0546] An "device" is a mechanical or system component designed to perform a specific function or role.

[0547] "Analysis" is the process of breaking down data and information to understand their meaning and structure.

[0548] "Intention" refers to the goals and desires that a user tries to achieve based on their own hopes and objectives.

[0549] An "information collection" is a systematic database or repository that organizes and manages diverse data and information.

[0550] "Searching" is the act of searching for data in order to find the information or results that are being sought.

[0551] A "product" is an object that is sold as a tangible item or an intangible service.

[0552] "Presentation" refers to the act of showing information or suggestions to a user and prompting them to make a choice or decision.

[0553] For this invention to be implemented, coordination between the server and the user terminal is crucial. The server receives information entered by the user in natural language and processes it to analyze its context and keywords. Natural language processing technologies such as the Google Cloud NLP API are used for the analysis to extract the input content as specific requests and intentions.

[0554] The extracted information is then passed on to a process that searches for relevant product data in a database, which is a collection of information. This database search uses a high-speed data management system, such as Firebase. The resulting product list is then presented to the user's device, which could be a smartphone or a personal computer.

[0555] Users can select their desired items from the presented products, and the server proceeds with the purchase process based on their selection. An electronic payment platform is integrated into the purchase process to support smooth transaction completion.

[0556] For example, if a user enters "What do I need for a healthy breakfast?", the server will begin analyzing the request and suggest corresponding ingredients and recipes. Furthermore, if the user enters a prompt such as "What foods are recommended for a weekend picnic?", related products such as sandwiches and snacks will be listed. Throughout this entire process, users can easily and efficiently obtain the products and services they need.

[0557] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0558] Step 1:

[0559] The user inputs their request into the device using natural language. For example, they might text or voice a request saying, "Tell me what I need for a healthy breakfast." The device receives this input and prepares to send it to the server. Regardless of the format or type of input, accurately conveying the user's intent is crucial.

[0560] Step 2:

[0561] The server receives natural language input from the user via the terminal. Next, it analyzes the input text using natural language processing technologies such as the Google Cloud NLP API. During the analysis process, keywords are extracted to understand the user's request and grasp the context. This analysis outputs relevant phrases such as "healthy breakfast" and "needs."

[0562] Step 3:

[0563] Based on the analysis results, the server searches databases such as Firebase. The previously extracted keywords are used as search queries to find relevant product information. The information retrieved from the database is output as a list of products that match the user's request.

[0564] Step 4:

[0565] The server returns the retrieved product list to the terminal. The terminal receives this list and presents it visually to the user. The list includes detailed information such as product name, description, and price, which the user can compare to increase their purchase intent.

[0566] Step 5:

[0567] Once the user selects the items they wish to purchase, the server receives this selection and initiates the purchase process. Based on the information of the selected items, it prepares to execute the transaction via the electronic payment platform. The purpose of this step is to complete the transaction securely and quickly.

[0568] Step 6:

[0569] After a transaction is completed, the server records the details of the purchase and notifies the service provider of the information. Finally, the server aggregates and manages this information to ensure transaction transparency and efficient management.

[0570] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0571] The concierge AI system incorporating the emotion engine according to the present invention analyzes the user's natural language input and further recognizes the user's emotions from the input to provide more personalized services. This system includes an advanced process for making appropriate and effective service suggestions in response to the user's input.

[0572] Users input their wishes and requests into the system via text or voice through their device. The emotion engine then analyzes the input information to identify emotional nuances. As a result, the type of emotion the user is experiencing—such as joy, sadness, surprise, or anger—becomes clear.

[0573] The terminal sends user input data to the server, which uses natural language processing (NLP) to analyze the input in detail. This extracts the user's specific intentions. Furthermore, emotional information recognized by the emotion engine is also processed by the server, and an appropriate service category is set according to the user's state.

[0574] The server searches the database based on the analysis results and emotional information to identify relevant service candidates. During this process, suggestions are made that take the user's emotional state into account, and the priority of recommended services is adjusted based on emotional data. This process might, for example, suggest services suitable for relaxation if the user is stressed, or more stimulating services if they are excited.

[0575] For example, if a user types "Tell me how to relieve stress," the server will respond by extracting keywords such as "stress relief" and "methods." The emotion engine senses the user's stress level from their voice tone and text expression, and suggests entertainment, health-related services, or events that are most appropriate for that state.

[0576] The list of suggestions sent to the terminal is displayed to the user, who can then make a selection based on it. The introduction of an emotion engine enables suggestions that harmonize with the user's intuition and state of mind, further improving the user experience. The server executes the service purchase procedure according to the user's selection and manages the final revenue. In this way, the system utilizing the emotion engine provides a more refined service experience by simultaneously considering the user's emotions and needs.

[0577] The following describes the processing flow.

[0578] Step 1:

[0579] The user enters their wishes and requests into the terminal, and input can be done via text or voice. The terminal then retrieves the input data and prepares for the next process.

[0580] Step 2:

[0581] The device sends the acquired input data to the server. At the same time, in the case of voice input, it prepares the emotion engine to start the emotion analysis process.

[0582] Step 3:

[0583] The server analyzes the received natural language data using a natural language processing (NLP) module. This analysis extracts keywords and phrases related to the user's intent.

[0584] Step 4:

[0585] The server uses an emotion engine to recognize the user's emotions from the input data. The analyzed emotion information is used to identify the user's emotional state (e.g., joy, sadness, surprise, anger).

[0586] Step 5:

[0587] The server searches the database for relevant services based on the analyzed intent and emotional information. Based on the emotional information, the service candidates that best match the user's mental state are selected preferentially.

[0588] Step 6:

[0589] The server organizes the services obtained from the search results and creates a suggested list that is tailored based on the user's feelings and intentions. This suggested list contains the services that are most suitable for the user.

[0590] Step 7:

[0591] The terminal displays a list of suggestions sent from the server to the user. The user can then select the service that best suits their needs from the suggested options.

[0592] Step 8:

[0593] When a user selects a specific service, the device sends that information back to the server. The server then initiates the purchase or reservation process.

[0594] Step 9:

[0595] The server executes the purchase process for the selected service and completes the requested actions. Finally, the server receives and records the revenue from the service provider.

[0596] (Example 2)

[0597] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0598] Conventional information provision systems only suggest services based on user input, without considering the user's emotional state. Therefore, they were unable to provide personalized services tailored to the user's emotions, resulting in insufficient improvement in user satisfaction.

[0599] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0600] In this invention, the server includes means for receiving natural language input from a user, means for analyzing the input to identify the user's intent, and means for analyzing the emotions contained in the input using an emotion engine. This enables personalized service suggestions based on the user's emotional state.

[0601] A "user" refers to a person who provides input to the system using natural language.

[0602] "Natural language" refers to the words and sentences that humans use in everyday life, and is information that should be processed in order to be understood by computers.

[0603] "Input" refers to information that a user provides to the system in the form of text or voice.

[0604] "Analysis" is the process of examining input natural language syntactically and semantically and converting the information into a format that is easy to understand.

[0605] "Intention" refers to the purpose or request that the user is trying to convey through their input.

[0606] An "emotion engine" is a technology used to identify emotional nuances from user input.

[0607] A "database" refers to a searchable storage system that organizes and stores related information and data.

[0608] A "generative AI model" is a system that uses machine learning to automatically generate responses and suggestions based on input data.

[0609] "Suggestions" refer to potentially useful information or services provided to the user as a result of analysis and database searches.

[0610] "Purchase process" refers to all operations and arrangements necessary to enable the user to use the service they have selected.

[0611] "Revenue" refers to the financial profit that the system operator receives as payment for the services provided.

[0612] In this invention, the user uses a terminal to perform natural language input. The input is sent to the server in the form of text or voice. The server utilizes a natural language processing engine to analyze the received input data and identify the user's intent. In this process, the grammar and meaning of the input are examined in detail to understand what the user's request is.

[0613] Next, the server uses an emotion engine to analyze the emotions contained in the user's input. This engine can identify nuances of emotion based on the tone of voice and text expression. As a result of the analysis, the user's emotional state is classified into categories such as joy, sadness, surprise, and anger.

[0614] The server searches the database for relevant information based on the user's intent and emotional information. This database stores a variety of services and suggestions, and searches are performed according to the user's state.

[0615] Using a generative AI model, the server automatically generates service suggestions tailored to the user based on the extracted information. These suggestions are then sent to the terminal and displayed to the user. The user can then select the most appropriate service from the displayed options.

[0616] As a concrete example, consider a scenario where a user types "Tell me how to relieve stress" into their device. In this case, the server analyzes keywords such as "stress relief" and "methods." The emotion engine detects the user's stress level from their input. Based on this, the server uses a generating AI model to suggest music, exercise, or other relaxation-related activities. An example of a prompt could be, "Please suggest the most suitable entertainment service for when the user is feeling relaxed."

[0617] This system aims to improve user satisfaction by enabling suggestions that take into account both the user's emotions and intentions.

[0618] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0619] Step 1:

[0620] Users input information in natural language through their devices. Input can be in text or voice format. The input information is sent to the server in its original format. For example, if a user says to their smartphone, "I want to know how to relax," the voice data is recorded and converted into text.

[0621] Step 2:

[0622] The terminal sends the input natural language data to the server. The input is the user's voice or text data, and the output is data packets sent to the server via the network. The data is transferred in real time, allowing the process to proceed to the next step without delay.

[0623] Step 3:

[0624] The server passes the received data to the natural language processing engine. The input is the user's text data, and the output is information about the analyzed grammatical structure and the user's intent. The natural language processing engine performs grammatical analysis on the input data and extracts keywords such as "relax" and "method."

[0625] Step 4:

[0626] The server activates an emotion engine to analyze the user's emotions from their input. The input is text data such as "I want to know how to relax," and the output is emotional information such as "The user is feeling stressed." In the case of voice input, the tone of voice is also used for emotion analysis.

[0627] Step 5:

[0628] The server searches the database based on the analyzed intent and sentiment information. The input is the analyzed information regarding intent and sentiment, and the output is information on multiple related services. The database search finds the most suitable service based on specific keywords or sentiments.

[0629] Step 6:

[0630] Using a generative AI model, the server generates suggestions for the user. The input consists of service information retrieved from a database and the user's emotional state, while the output is a specific service suggestion. The prompt used is "Suggest the most suitable entertainment service for when the user is feeling relaxed."

[0631] Step 7:

[0632] The terminal displays suggestions. The output from the server is a list of specific services displayed on the user's screen, and the input is suggestion data sent from the server. The user selects a service based on this list.

[0633] Step 8:

[0634] Based on the service selected by the user, the server executes the purchase process. The input is the user's selection data, and the output is purchase information sent to the service provider and a confirmation message to the user. Specific actions include confirming purchase information, executing payment, and confirming the service reservation.

[0635] (Application Example 2)

[0636] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0637] Traditional personalized service delivery systems can analyze user intent, but they have a challenge in that they do not adequately provide suggestions based on user emotions. Therefore, it is necessary to improve the user experience by suggesting appropriate services according to the user's emotional state.

[0638] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0639] In this invention, the server includes means for receiving natural language input from a user, means for analyzing the natural language input to identify the user's intent and emotional information, and means for searching a database for relevant services based on the user's intent and emotional information. This enables personalized service suggestions that correspond to the user's emotional state.

[0640] "Natural language input" refers to a method of inputting words that users use in their daily lives directly into the system.

[0641] "Analysis" is the process of examining input information in detail and interpreting its meaning and intent.

[0642] "User intent" refers to the requirements and desires that users have for the system.

[0643] "Emotional information" refers to data about a user's psychological state that can be gleaned from their statements and text.

[0644] A "database" is a structured collection of data used to efficiently store, retrieve, and manage specific information.

[0645] "Personalized suggestions" refers to the provision of services that are individually customized to suit each user's characteristics and circumstances.

[0646] "Priority" refers to the order in which multiple options are considered important or should be dealt with first.

[0647] The "purchase process" is a series of operations that transfer ownership of the selected service to the user.

[0648] "Revenue" refers to the monetary profit obtained from the service provider.

[0649] This invention configures a system in a specific way to process natural language input from a user. When a user inputs instructions into the terminal via speech or text, the terminal uses a speech recognition API to convert the speech into text data. Next, a natural language processing library is used to analyze the text data and extract the user's intent and sentiment information.

[0650] The server searches a large database for relevant services based on extracted intent and sentiment information. It further analyzes the emotional state using a sentiment analysis API to determine the priority of services best suited to the user's emotions. This enables more personalized recommendations.

[0651] This result can be returned to the device, displaying the most suitable service suggestions to the user based on their emotions. Based on the suggested service, the purchase process is executed, and revenue from the service provider is managed simultaneously. This revenue management ensures mutually beneficial transactions for both the user and the service provider.

[0652] As a concrete example, if a user types "Tell me how to relax" into their device, the system uses an emotion engine to identify the user's stress level. Based on this, it can suggest things like meditation apps, relaxing music, or guided yoga classes. An example of a prompt to the related generative AI model would be, "The user is seeking relaxation. Please list suggestions that can help relieve stress, such as music, aromatherapy, and yoga classes."

[0653] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0654] Step 1:

[0655] The user provides input to the device via voice or text. If voice input is received, the device uses a speech recognition API to convert it into text data. The input data may include the user's desire to relax.

[0656] Step 2:

[0657] The server analyzes the text data received from the terminal using a natural language processing library. This analysis extracts the user's intent as "seeking relaxation." The output provides keywords related to the user's intent.

[0658] Step 3:

[0659] The server uses a sentiment analysis API to analyze the sentiment information contained in the text data. Emotions (e.g., stress) are identified from words and phrases in the input data. The resulting sentiment information is then output.

[0660] Step 4:

[0661] The server searches the database based on the extracted intent and emotional information. This search process filters services and content related to "relaxation," listing the most suitable service candidates for that emotion. The output is a prioritized list of services.

[0662] Step 5:

[0663] The terminal displays a list of service options sent from the server to the user. The user can then select a service of interest from this list. This executes a selection action on the user interface.

[0664] Step 6:

[0665] The server executes the purchase process for the service selected by the user. The input is the service information selected by the user, and the output generates purchase confirmations and payment completion notifications. The server receives and manages revenue from the service provider.

[0666] Step 7:

[0667] The server sends a prompt message to the generative AI model based on the service suggestions. Specifically, a prompt message like, "The user is seeking relaxation. Please list suggestions that can help relieve stress, such as music, aromatherapy, and yoga classes," is generated. This allows the generative AI model to update its database to provide more personalized services.

[0668] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0669] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0670] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0671] [Fourth Embodiment]

[0672] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0673] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0674] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0675] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0676] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0677] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0678] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0679] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0680] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0681] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0682] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0683] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0684] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0685] The concierge AI system according to the present invention provides a function that analyzes the user's natural language input and suggests relevant services. This system includes a series of processes to simplify user operation and facilitate the selection of the optimal service.

[0686] Users can access the system via devices such as smartphones and PCs and input their needs and preferences in natural language. This input can be in text or voice, providing a flexible interface. The device sends the user input to the server, and the analysis process begins.

[0687] The server analyzes the received natural language input using a natural language processing (NLP) module. This analysis converts the user's intent into specific keywords and phrases, helping to identify the services the user truly desires. The NLP results are recognized as categories and actions that align with the user's wishes.

[0688] Next, the server searches the database based on the analyzed information to identify services that match the user's intent. This search process takes into account factors such as the service provider, features, availability, and cost. If multiple services are identified as search results, the optimal service candidates are listed.

[0689] For example, if a user enters "I want to go on a trip this weekend," the server analyzes this and extracts the keywords "travel" and "weekend." It then searches the database for travel-related service information and generates a list of suitable travel packages and accommodations. The user can then view this list on their device screen and select the services that interest them.

[0690] When a user selects a service, the terminal sends that information back to the server, which then begins the purchase process. Once the purchase is confirmed, the transaction is completed, the server receives the revenue from the service provider, and notifies the system operator or administrator.

[0691] In this way, the concierge AI system can simplify the process for users to quickly select the appropriate service, thereby maximizing its effectiveness.

[0692] The following describes the processing flow.

[0693] Step 1:

[0694] Users input their wishes and requests in natural language using the device. Input can be done via text or voice, and the device acquires this as digital data.

[0695] Step 2:

[0696] The terminal sends the acquired user input data to the server. This transmission takes place in real time via the internet.

[0697] Step 3:

[0698] The server passes the received natural language data to a natural language processing (NLP) module. Within the server, this data is tokenized, and important keywords and phrases are extracted.

[0699] Step 4:

[0700] The server identifies relevant categories and actions based on the user's intent identified by NLP. This clarifies the information that addresses the user's needs.

[0701] Step 5:

[0702] The server searches the database for services that match the identified intent. This search prioritizes highly relevant services and also takes into account the service's characteristics and terms of service.

[0703] Step 6:

[0704] The server generates a list of the most suitable services to suggest to the user based on the search results. This list is structured to best match the user's preferences.

[0705] Step 7:

[0706] The terminal displays a list of service suggestions sent from the server to the user. The user can review this list and select the service they like.

[0707] Step 8:

[0708] When a user selects a service, the device sends that selection information back to the server. This selection information includes basic data necessary for the purchase.

[0709] Step 9:

[0710] After receiving the user's selection information, the server executes the necessary purchase procedures. This includes service reservations and payment processing.

[0711] Step 10:

[0712] The server notifies the corresponding service provider after the purchase process is complete and calculates and manages the revenue. Revenue from the service provider is recorded and later reported to the operator.

[0713] (Example 1)

[0714] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0715] Traditional information provision systems required users to spend a great deal of time and effort searching for the information and services they needed, resulting in a poor user experience during the selection and purchase process. Furthermore, the lack of technology to effectively utilize voice input and accurately grasp user intent using natural language meant that user needs could not be fully met.

[0716] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0717] In this invention, the server includes a device for receiving natural language input from a user, a processing device for analyzing the natural language input and identifying the user's intent, and a computing device for retrieving relevant information from a storage device based on the user's intent. This enables the user to quickly and accurately obtain information and services that match their needs and enjoy a comfortable selection and purchasing experience.

[0718] A "device" is a machine or electronic device designed to perform a specific function.

[0719] A "processing device" is a computer or dedicated hardware that analyzes input information and performs calculations or operations to execute a specified task.

[0720] A "storage device" is a piece of hardware or system used to store data and information, and to retrieve and use it as needed.

[0721] A "processing unit" is a mechanism or electronic circuit that performs calculations and logical decisions based on input data and outputs the results.

[0722] A "transmitting device" is a machine or program that has the function of transmitting information to another device or system.

[0723] A "receiving device" is an electronic device or system that receives, analyzes, or displays information transmitted from an external source.

[0724] A "conversion device" is a system or circuit designed to change data from one format to another.

[0725] An "analysis device" is hardware or software used to structure information and break it down or understand it according to a specific purpose.

[0726] A "management device" refers to equipment or a program that has the function of monitoring and controlling a system or process.

[0727] This invention is a system that quickly suggests relevant information and services based on information entered by the user in natural language. The system consists of a server, a terminal, and a user, and aims to improve the overall user experience.

[0728] User actions

[0729] Users access the system using devices such as smartphones and PCs. Here, they can input their needs and preferences using natural language, i.e., text or voice. If voice input is selected, the device has a built-in conversion device that converts speech to text. This function utilizes speech recognition technology, enabling efficient text conversion of voice input.

[0730] Terminal processing

[0731] The terminal sends user input to the server. Encrypted communication is often used for data transmission to ensure security and speed. This transmission allows the server to receive the user's request.

[0732] Server-based processing

[0733] The server uses an NLP (Natural Language Processing) module to analyze user input. This module converts the input into important keywords and phrases to understand the user's intent. Common APIs and libraries may be used for this analysis.

[0734] Based on the analysis results, the server searches for relevant information stored in its memory. A database management system is used to efficiently find services that meet the user's needs. For example, if a user enters "I want to go on a trip this weekend," the server analyzes this and extracts keywords such as "travel" and "weekend."

[0735] Service proposal and purchase procedure

[0736] The server sends information to the user's terminal to suggest multiple options obtained through the search. The user can review these suggestions on the screen and select those that interest them. An example of a suggestion message would be: "The user is looking for a weekend trip. Please suggest recommended travel packages."

[0737] Once the user makes a selection and decides to purchase the service, the device sends that information back to the server. The server initiates the purchase process and uses the payment system to ensure the transaction is completed securely. Once payment is complete, the system receives the payment from the supplier and notifies the administrator of this information, thus completing the entire process.

[0738] In this way, the system provides users with information quickly and efficiently, and also provides an environment that allows for a smooth purchasing process.

[0739] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0740] Step 1:

[0741] The user uses the device to input natural language. For example, the user might input text or voice, such as "I want to go on a trip this weekend." The device receives this user input and converts the voice to text if necessary. The resulting input data is natural language text.

[0742] Step 2:

[0743] The terminal sends the obtained text to the server. A secure communication protocol is used for transmission. The server receives natural language text as input data from the terminal's output.

[0744] Step 3:

[0745] The server processes the received natural language text through a natural language processing (NLP) module. At this stage, the server analyzes the input data and identifies the user's intent. Specifically, keywords such as "travel" and "weekend" are extracted. This results in the intent analysis results being output.

[0746] Step 4:

[0747] The server searches its internal database based on the intent analysis results. The server retrieves information related to the keywords from its storage and finds services that match the user's needs. The output is a list of search results.

[0748] Step 5:

[0749] The server sends a list of search results to the device. The list includes multiple services and products that match the user's preferences. The device converts the received data into a UI / UX format for display to the user.

[0750] Step 6:

[0751] The user selects a service of interest from a list of options displayed on the device screen. Once a selection is made, the device sends this selection information to the server. This is where data about the user's selected service is input.

[0752] Step 7:

[0753] The server initiates the purchase process based on the user's selection, integrating with an external payment system to process the transaction. Once payment is confirmed, the server acknowledges the transaction's completion and notifies both the user and the service provider that the transaction was successful. A purchase confirmation notification is generated as output.

[0754] (Application Example 1)

[0755] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0756] In modern e-commerce, it is not easy for users to find the best products for themselves in a short amount of time. Furthermore, there is a need for efficient methods to provide appropriate suggestions from a vast amount of product information and to facilitate the purchase process smoothly. Traditional systems often have a gap between user demand and the products offered, which can easily diminish purchasing intent.

[0757] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0758] In this invention, the server includes a device for receiving natural language input from a user, a device for analyzing the natural language input to identify the user's intent, and a device for searching for relevant products from a set of information based on the user's intent. This makes it possible to quickly and accurately present product information that meets the user's requirements.

[0759] A "user" is someone who utilizes the system, inputting their own wishes and requests and receiving suggestions in response.

[0760] "Natural language" refers to the language system that humans use on a daily basis, and which is expressed in either spoken or written form.

[0761] "Input" refers to the act of a user providing information or data to a system.

[0762] An "device" is a mechanical or system component designed to perform a specific function or role.

[0763] "Analysis" is the process of breaking down data and information to understand their meaning and structure.

[0764] "Intention" refers to the goals and desires that a user tries to achieve based on their own hopes and objectives.

[0765] An "information collection" is a systematic database or repository that organizes and manages diverse data and information.

[0766] "Searching" is the act of searching for data in order to find the information or results that are being sought.

[0767] A "product" is an object that is sold as a tangible item or an intangible service.

[0768] "Presentation" refers to the act of showing information or suggestions to a user and prompting them to make a choice or decision.

[0769] For this invention to be implemented, coordination between the server and the user terminal is crucial. The server receives information entered by the user in natural language and processes it to analyze its context and keywords. Natural language processing technologies such as the Google Cloud NLP API are used for the analysis to extract the input content as specific requests and intentions.

[0770] The extracted information is then passed on to a process that searches for relevant product data in a database, which is a collection of information. This database search uses a high-speed data management system, such as Firebase. The resulting product list is then presented to the user's device, which could be a smartphone or a personal computer.

[0771] Users can select their desired items from the presented products, and the server proceeds with the purchase process based on their selection. An electronic payment platform is integrated into the purchase process to support smooth transaction completion.

[0772] For example, if a user enters "What do I need for a healthy breakfast?", the server will begin analyzing the request and suggest corresponding ingredients and recipes. Furthermore, if the user enters a prompt such as "What foods are recommended for a weekend picnic?", related products such as sandwiches and snacks will be listed. Throughout this entire process, users can easily and efficiently obtain the products and services they need.

[0773] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0774] Step 1:

[0775] The user inputs their request into the device using natural language. For example, they might text or voice a request saying, "Tell me what I need for a healthy breakfast." The device receives this input and prepares to send it to the server. Regardless of the format or type of input, accurately conveying the user's intent is crucial.

[0776] Step 2:

[0777] The server receives natural language input from the user via the terminal. Next, it analyzes the input text using natural language processing technologies such as the Google Cloud NLP API. During the analysis process, keywords are extracted to understand the user's request and grasp the context. This analysis outputs relevant phrases such as "healthy breakfast" and "needs."

[0778] Step 3:

[0779] Based on the analysis results, the server searches databases such as Firebase. The previously extracted keywords are used as search queries to find relevant product information. The information retrieved from the database is output as a list of products that match the user's request.

[0780] Step 4:

[0781] The server returns the retrieved product list to the terminal. The terminal receives this list and presents it visually to the user. The list includes detailed information such as product name, description, and price, which the user can compare to increase their purchase intent.

[0782] Step 5:

[0783] Once the user selects the items they wish to purchase, the server receives this selection and initiates the purchase process. Based on the information of the selected items, it prepares to execute the transaction via the electronic payment platform. The purpose of this step is to complete the transaction securely and quickly.

[0784] Step 6:

[0785] After a transaction is completed, the server records the details of the purchase and notifies the service provider of the information. Finally, the server aggregates and manages this information to ensure transaction transparency and efficient management.

[0786] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0787] The concierge AI system incorporating the emotion engine according to the present invention analyzes the user's natural language input and further recognizes the user's emotions from the input to provide more personalized services. This system includes an advanced process for making appropriate and effective service suggestions in response to the user's input.

[0788] Users input their wishes and requests into the system via text or voice through their device. The emotion engine then analyzes the input information to identify emotional nuances. As a result, the type of emotion the user is experiencing—such as joy, sadness, surprise, or anger—becomes clear.

[0789] The terminal sends user input data to the server, which uses natural language processing (NLP) to analyze the input in detail. This extracts the user's specific intentions. Furthermore, emotional information recognized by the emotion engine is also processed by the server, and an appropriate service category is set according to the user's state.

[0790] The server searches the database based on the analysis results and emotional information to identify relevant service candidates. During this process, suggestions are made that take the user's emotional state into account, and the priority of recommended services is adjusted based on emotional data. This process might, for example, suggest services suitable for relaxation if the user is stressed, or more stimulating services if they are excited.

[0791] For example, if a user types "Tell me how to relieve stress," the server will respond by extracting keywords such as "stress relief" and "methods." The emotion engine senses the user's stress level from their voice tone and text expression, and suggests entertainment, health-related services, or events that are most appropriate for that state.

[0792] The list of suggestions sent to the terminal is displayed to the user, who can then make a selection based on it. The introduction of an emotion engine enables suggestions that harmonize with the user's intuition and state of mind, further improving the user experience. The server executes the service purchase procedure according to the user's selection and manages the final revenue. In this way, the system utilizing the emotion engine provides a more refined service experience by simultaneously considering the user's emotions and needs.

[0793] The following describes the processing flow.

[0794] Step 1:

[0795] The user enters their wishes and requests into the terminal, and input can be done via text or voice. The terminal then retrieves the input data and prepares for the next process.

[0796] Step 2:

[0797] The device sends the acquired input data to the server. At the same time, in the case of voice input, it prepares the emotion engine to start the emotion analysis process.

[0798] Step 3:

[0799] The server analyzes the received natural language data using a natural language processing (NLP) module. This analysis extracts keywords and phrases related to the user's intent.

[0800] Step 4:

[0801] The server uses an emotion engine to recognize the user's emotions from the input data. The analyzed emotion information is used to identify the user's emotional state (e.g., joy, sadness, surprise, anger).

[0802] Step 5:

[0803] The server searches the database for relevant services based on the analyzed intent and emotional information. Based on the emotional information, the service candidates that best match the user's mental state are selected preferentially.

[0804] Step 6:

[0805] The server organizes the services obtained from the search results and creates a suggested list that is tailored based on the user's feelings and intentions. This suggested list contains the services that are most suitable for the user.

[0806] Step 7:

[0807] The terminal displays a list of suggestions sent from the server to the user. The user can then select the service that best suits their needs from the suggested options.

[0808] Step 8:

[0809] When a user selects a specific service, the device sends that information back to the server. The server then initiates the purchase or reservation process.

[0810] Step 9:

[0811] The server executes the purchase process for the selected service and completes the requested actions. Finally, the server receives and records the revenue from the service provider.

[0812] (Example 2)

[0813] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0814] Conventional information provision systems only suggest services based on user input, without considering the user's emotional state. Therefore, they were unable to provide personalized services tailored to the user's emotions, resulting in insufficient improvement in user satisfaction.

[0815] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0816] In this invention, the server includes means for receiving natural language input from a user, means for analyzing the input to identify the user's intent, and means for analyzing the emotions contained in the input using an emotion engine. This enables personalized service suggestions based on the user's emotional state.

[0817] A "user" refers to a person who provides input to the system using natural language.

[0818] "Natural language" refers to the words and sentences that humans use in everyday life, and is information that should be processed in order to be understood by computers.

[0819] "Input" refers to information that a user provides to the system in the form of text or voice.

[0820] "Analysis" is the process of examining input natural language syntactically and semantically and converting the information into a format that is easy to understand.

[0821] "Intention" refers to the purpose or request that the user is trying to convey through their input.

[0822] An "emotion engine" is a technology used to identify emotional nuances from user input.

[0823] A "database" refers to a searchable storage system that organizes and stores related information and data.

[0824] A "generative AI model" is a system that uses machine learning to automatically generate responses and suggestions based on input data.

[0825] "Suggestions" refer to potentially useful information or services provided to the user as a result of analysis and database searches.

[0826] "Purchase process" refers to all operations and arrangements necessary to enable the user to use the service they have selected.

[0827] "Revenue" refers to the financial profit that the system operator receives as payment for the services provided.

[0828] In this invention, the user uses a terminal to perform natural language input. The input is sent to the server in the form of text or voice. The server utilizes a natural language processing engine to analyze the received input data and identify the user's intent. In this process, the grammar and meaning of the input are examined in detail to understand what the user's request is.

[0829] Next, the server uses an emotion engine to analyze the emotions contained in the user's input. This engine can identify nuances of emotion based on the tone of voice and text expression. As a result of the analysis, the user's emotional state is classified into categories such as joy, sadness, surprise, and anger.

[0830] The server searches the database for relevant information based on the user's intent and emotional information. This database stores a variety of services and suggestions, and searches are performed according to the user's state.

[0831] Using a generative AI model, the server automatically generates service suggestions tailored to the user based on the extracted information. These suggestions are then sent to the terminal and displayed to the user. The user can then select the most appropriate service from the displayed options.

[0832] As a concrete example, consider a scenario where a user types "Tell me how to relieve stress" into their device. In this case, the server analyzes keywords such as "stress relief" and "methods." The emotion engine detects the user's stress level from their input. Based on this, the server uses a generating AI model to suggest music, exercise, or other relaxation-related activities. An example of a prompt could be, "Please suggest the most suitable entertainment service for when the user is feeling relaxed."

[0833] This system aims to improve user satisfaction by enabling suggestions that take into account both the user's emotions and intentions.

[0834] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0835] Step 1:

[0836] Users input information in natural language through their devices. Input can be in text or voice format. The input information is sent to the server in its original format. For example, if a user says to their smartphone, "I want to know how to relax," the voice data is recorded and converted into text.

[0837] Step 2:

[0838] The terminal sends the input natural language data to the server. The input is the user's voice or text data, and the output is data packets sent to the server via the network. The data is transferred in real time, allowing the process to proceed to the next step without delay.

[0839] Step 3:

[0840] The server passes the received data to the natural language processing engine. The input is the user's text data, and the output is information about the analyzed grammatical structure and the user's intent. The natural language processing engine performs grammatical analysis on the input data and extracts keywords such as "relax" and "method."

[0841] Step 4:

[0842] The server activates an emotion engine to analyze the user's emotions from their input. The input is text data such as "I want to know how to relax," and the output is emotional information such as "The user is feeling stressed." In the case of voice input, the tone of voice is also used for emotion analysis.

[0843] Step 5:

[0844] The server searches the database based on the analyzed intent and sentiment information. The input is the analyzed information regarding intent and sentiment, and the output is information on multiple related services. The database search finds the most suitable service based on specific keywords or sentiments.

[0845] Step 6:

[0846] Using a generative AI model, the server generates suggestions for the user. The input consists of service information retrieved from a database and the user's emotional state, while the output is a specific service suggestion. The prompt used is "Suggest the most suitable entertainment service for when the user is feeling relaxed."

[0847] Step 7:

[0848] The terminal displays suggestions. The output from the server is a list of specific services displayed on the user's screen, and the input is suggestion data sent from the server. The user selects a service based on this list.

[0849] Step 8:

[0850] Based on the service selected by the user, the server executes the purchase process. The input is the user's selection data, and the output is purchase information sent to the service provider and a confirmation message to the user. Specific actions include confirming purchase information, executing payment, and confirming the service reservation.

[0851] (Application Example 2)

[0852] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0853] Traditional personalized service delivery systems can analyze user intent, but they have a challenge in that they do not adequately provide suggestions based on user emotions. Therefore, it is necessary to improve the user experience by suggesting appropriate services according to the user's emotional state.

[0854] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0855] In this invention, the server includes means for receiving natural language input from a user, means for analyzing the natural language input to identify the user's intent and emotional information, and means for searching a database for relevant services based on the user's intent and emotional information. This enables personalized service suggestions that correspond to the user's emotional state.

[0856] "Natural language input" refers to a method of inputting words that users use in their daily lives directly into the system.

[0857] "Analysis" is the process of examining input information in detail and interpreting its meaning and intent.

[0858] "User intent" refers to the requirements and desires that users have for the system.

[0859] "Emotional information" refers to data about a user's psychological state that can be gleaned from their statements and text.

[0860] A "database" is a structured collection of data used to efficiently store, retrieve, and manage specific information.

[0861] "Personalized suggestions" refers to the provision of services that are individually customized to suit each user's characteristics and circumstances.

[0862] "Priority" refers to the order in which multiple options are considered important or should be dealt with first.

[0863] The "purchase process" is a series of operations that transfer ownership of the selected service to the user.

[0864] "Revenue" refers to the monetary profit obtained from the service provider.

[0865] This invention configures a system in a specific way to process natural language input from a user. When a user inputs instructions into the terminal via speech or text, the terminal uses a speech recognition API to convert the speech into text data. Next, a natural language processing library is used to analyze the text data and extract the user's intent and sentiment information.

[0866] The server searches a large database for relevant services based on extracted intent and sentiment information. It further analyzes the emotional state using a sentiment analysis API to determine the priority of services best suited to the user's emotions. This enables more personalized recommendations.

[0867] This result can be returned to the device, displaying the most suitable service suggestions to the user based on their emotions. Based on the suggested service, the purchase process is executed, and revenue from the service provider is managed simultaneously. This revenue management ensures mutually beneficial transactions for both the user and the service provider.

[0868] As a concrete example, if a user types "Tell me how to relax" into their device, the system uses an emotion engine to identify the user's stress level. Based on this, it can suggest things like meditation apps, relaxing music, or guided yoga classes. An example of a prompt to the related generative AI model would be, "The user is seeking relaxation. Please list suggestions that can help relieve stress, such as music, aromatherapy, and yoga classes."

[0869] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0870] Step 1:

[0871] The user provides input to the device via voice or text. If voice input is received, the device uses a speech recognition API to convert it into text data. The input data may include the user's desire to relax.

[0872] Step 2:

[0873] The server analyzes the text data received from the terminal using a natural language processing library. This analysis extracts the user's intent as "seeking relaxation." The output provides keywords related to the user's intent.

[0874] Step 3:

[0875] The server uses a sentiment analysis API to analyze the sentiment information contained in the text data. Emotions (e.g., stress) are identified from words and phrases in the input data. The resulting sentiment information is then output.

[0876] Step 4:

[0877] The server searches the database based on the extracted intent and emotional information. This search process filters services and content related to "relaxation," listing the most suitable service candidates for that emotion. The output is a prioritized list of services.

[0878] Step 5:

[0879] The terminal displays a list of service options sent from the server to the user. The user can then select a service of interest from this list. This executes a selection action on the user interface.

[0880] Step 6:

[0881] The server executes the purchase process for the service selected by the user. The input is the service information selected by the user, and the output generates purchase confirmations and payment completion notifications. The server receives and manages revenue from the service provider.

[0882] Step 7:

[0883] The server sends a prompt message to the generative AI model based on the service suggestions. Specifically, a prompt message like, "The user is seeking relaxation. Please list suggestions that can help relieve stress, such as music, aromatherapy, and yoga classes," is generated. This allows the generative AI model to update its database to provide more personalized services.

[0884] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0885] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0886] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0887] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0888] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0889] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0890] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0891] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0892] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0893] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0894] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0895] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0896] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0897] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0898] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0899] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0900] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0901] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0902] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0903] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0904] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0905] The following is further disclosed regarding the embodiments described above.

[0906] (Claim 1)

[0907] A means of receiving natural language input from users,

[0908] A means for analyzing the aforementioned natural language input to identify the user's intent,

[0909] A means for searching for relevant services from a database based on the user's intent,

[0910] A means for generating suggestions for the user from the aforementioned searched services,

[0911] A means for the user to carry out the purchase procedure for the service selected based on the above proposal,

[0912] A means of receiving revenue from the service provider for the purchase of the aforementioned service,

[0913] A system that includes this.

[0914] (Claim 2)

[0915] The system according to claim 1, including voice input.

[0916] (Claim 3)

[0917] The system according to claim 1, further comprising means for recording and managing revenue from service providers.

[0918] "Example 1"

[0919] (Claim 1)

[0920] A device that accepts natural language input from users,

[0921] A processing device that analyzes the aforementioned natural language input to identify the user's intent,

[0922] A computing device that retrieves relevant information from a storage device based on the user's intent,

[0923] A transmission device that generates suggestions for the user from the aforementioned searched information,

[0924] A processing device that performs the purchase procedure for information selected by the user based on the above proposal,

[0925] A receiving device that receives payment from the supplier for the purchase of the aforementioned information,

[0926] A conversion device that converts voice input to text,

[0927] An analysis device that analyzes information using a natural language processing module,

[0928] A system that includes this.

[0929] (Claim 2)

[0930] The system according to claim 1, further comprising a display device that uses a storage device to display information according to the user's selection.

[0931] (Claim 3)

[0932] The system according to claim 1, further comprising a management device for recording and managing payments from suppliers.

[0933] "Application Example 1"

[0934] (Claim 1)

[0935] A device that accepts natural language input from users,

[0936] A device that analyzes the aforementioned natural language input to identify the user's intent,

[0937] A device that searches for relevant products from a collection of information based on the user's intent,

[0938] A device that generates a presentation for the user from the searched products,

[0939] A device that executes the purchase procedure for the product selected by the user based on the aforementioned presentation,

[0940] A device for receiving a profit from the supplier for the purchase of the aforementioned product,

[0941] A device that dynamically presents relevant information about selected products to the user,

[0942] A system that includes this.

[0943] (Claim 2)

[0944] The system according to claim 1, including voice input.

[0945] (Claim 3)

[0946] The system according to claim 1, further comprising a device for recording and managing profits from a provider.

[0947] "Example 2 of combining an emotion engine"

[0948] (Claim 1)

[0949] A means of receiving natural language input from users,

[0950] A means for analyzing the aforementioned natural language input to identify the user's intent,

[0951] A means for analyzing the emotions contained in the input using an emotion engine,

[0952] A means for searching a database for relevant services based on the user's intent and emotional information,

[0953] A means for generating suggestions to the user from the searched services using a generative AI model,

[0954] A means for the user to carry out the purchase procedure for the service selected based on the above proposal,

[0955] A means of receiving revenue from the service provider for the purchase of the aforementioned service,

[0956] A system that includes this.

[0957] (Claim 2)

[0958] The system according to claim 1, including voice input.

[0959] (Claim 3)

[0960] The system according to claim 1, further comprising means for recording and managing revenue from service providers.

[0961] "Application example 2 when combining with an emotional engine"

[0962] (Claim 1)

[0963] A means of receiving natural language input from users,

[0964] A means for analyzing the aforementioned natural language input to identify the user's intent,

[0965] A means for searching a database for relevant services based on the user's intent and emotional information,

[0966] A means for generating suggestions from the aforementioned searched services in order of priority based on the user's emotional state,

[0967] A means for the user to carry out the purchase procedure for the service selected based on the above proposal,

[0968] A means of analyzing user emotions and providing personalized suggestions based on that state information,

[0969] A means of receiving revenue from the service provider for the purchase of the aforementioned service,

[0970] A system that includes this.

[0971] (Claim 2)

[0972] The system according to claim 1, including voice input.

[0973] (Claim 3)

[0974] The system according to claim 1, further comprising means for recording and managing revenue from service providers. [Explanation of Symbols]

[0975] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A device that accepts natural language input from users, A device that analyzes the aforementioned natural language input to identify the user's intent, A device that searches for relevant products from a collection of information based on the user's intent, A device that generates a presentation for the user from the searched products, A device that executes the purchase procedure for the product selected by the user based on the aforementioned presentation, A device for receiving a profit from the supplier for the purchase of the aforementioned product, A device that dynamically presents relevant information about selected products to the user, A system that includes this.

2. The system according to claim 1, including voice input.

3. The system according to claim 1, further comprising a device for recording and managing profits from a provider.

Images