system

Abstract

We provide the system. [Solution] A means for generating an artificial intelligence agent based on the consultation content entered by the user via an online platform, A means for selecting an appropriate expert from an expert database using the aforementioned artificial intelligence agent, A means of conducting real-time consultations between the user and a selected expert via the aforementioned artificial intelligence agent, After the aforementioned consultation is completed, a means of collecting feedback from the user and using it to improve the performance of the artificial intelligence agent, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Existing methods regarding consultations with experts have problems such as high economic burden, time constraints, and psychological hurdles. As a result, it has become difficult for many users to receive the necessary expert advice. Also, since it is difficult for experts themselves to handle many consultations within limited time, there is a problem that knowledge cannot be provided efficiently.

Means for Solving the Problems

[0005] This invention provides a system that generates an artificial intelligence agent via an online platform and selects an appropriate expert from a database of experts based on the content of the user's inquiry. The artificial intelligence agent analyzes the user's input using a natural language processing engine and generates an automated response according to the content of the inquiry. The expert database contains numerous expert profiles and is equipped with an algorithm that selects the most suitable expert based on their skills and experience. This makes it possible to provide an environment where experts can respond to more requests while reducing economic, time, and psychological hurdles for users.

[0006] An "online platform" is a digital infrastructure that allows users to access various services via the internet.

[0007] "User" refers to an individual or legal entity that wishes to consult with an expert through the system.

[0008] An "artificial intelligence agent" is a program that uses natural language processing technology to interact with users and automatically responds based on the content of their inquiry.

[0009] "Consultation details" refers to information such as problems or questions that users seek solutions for from experts.

[0010] A "specialist database" is a data aggregation system that collects profile information from multiple specialists and manages their skills and experience.

[0011] A "specialist" is an individual who possesses advanced knowledge and skills in a specific field and is qualified to provide advice to users.

[0012] A "natural language processing engine" is a computer program designed to understand and process human language, and is a technology used for analyzing text and speech.

[0013] "Automated response" refers to the answers or information that a system autonomously generates and provides in response to user input.

[0014] A "profile" is a dataset containing information about an expert's skills, experience, qualifications, and other relevant details.

[0015] "Skills" refer to specific techniques and abilities possessed by experts, and indicate the concrete abilities required for problem-solving.

[0016] An "algorithm" is a set of procedures or formulas for solving a specific problem, and in this context, it is used to select the appropriate expert from a database of experts. [Brief explanation of the drawing]

[0017] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10]Shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0018] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0019] First, the language used in the following description will be described.

[0020] In the following embodiments, a processor with a reference numeral (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of a plurality of arithmetic units. Further, the processor may be a single type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0021] In the following embodiments, a RAM (Random Access Memory) with a reference numeral is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0022] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0023] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0025] [First Embodiment]

[0026] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0027] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0030] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0033] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0037] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0038] As a means of implementing this invention, a system for consulting with experts using an online platform is constructed. The system consists of three main components: a server, a user's terminal, and an expert's device.

[0039] Server operation

[0040] The server plays a central role in this system, managing the database, generating artificial intelligence agents, and coordinating with experts. First, the server receives registered user information and stores it in the database. Then, based on the user's input, it uses a natural language processing engine to analyze the consultation content and generate an appropriate artificial intelligence agent. This agent facilitates smooth conversations with the user while utilizing information relevant to the consultation content to select the most suitable expert from the expert database.

[0041] Terminal operation

[0042] The user's device is an internet-connected device that, after registration, allows the user to consult with an artificial intelligence agent. Users can input consultation details related to their needs and receive responses in real time. If expert advice is needed, the AI ​​agent collects the information, queries the server, and provides appropriate real-time feedback.

[0043] Specific example

[0044] For example, if a user wants to "consult about a mortgage," the AI ​​agent understands the user's inquiry and sends that information to the server. The server searches a database of experts and selects an appropriate expert in the financial field. Based on the expert's registered profile, the AI ​​agent provides the user with expert advice and solutions. The user can easily check the agent's response on their device.

[0045] This specific example demonstrates that users can efficiently seek expert advice, while experts can create an efficient consultation environment that allows them to respond to a larger number of inquiries.

[0046] The following describes the processing flow.

[0047] Step 1:

[0048] The server verifies the user's access on the online platform and displays the account registration screen. The user enters their personal information and the area of ​​consultation they wish to pursue, and submits the registration form.

[0049] Step 2:

[0050] The server receives registration information submitted by the user, verifies the input, and then saves it to the database. After saving is complete, an automatic confirmation email is sent to the user to notify them of the registration completion.

[0051] Step 3:

[0052] The server invokes a natural language processing engine based on registered user information to generate an artificial intelligence agent that responds to the user's request. This agent includes an optimal language model based on the user's requirements.

[0053] Step 4:

[0054] The user logs into the system using their device and initiates a consultation through an artificial intelligence agent. The consultation details entered by the user are immediately analyzed by the agent.

[0055] Step 5:

[0056] The server searches a database of experts based on the consultation details received via the artificial intelligence agent. An algorithm then selects the expert best suited to the user's needs.

[0057] Step 6:

[0058] The server sends information on selected experts to an artificial intelligence agent, which then retrieves advice from the experts in real time and notifies the user.

[0059] Step 7:

[0060] The user reviews the advice provided by the agent on their device and asks further questions as needed. The agent continues to respond appropriately to these questions.

[0061] Step 8:

[0062] After the consultation ends, the server provides the user with a feedback format and stores the collected feedback in a database. This can then be used to improve future artificial intelligence agents.

[0063] (Example 1)

[0064] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0065] Conventional consultation systems have made it difficult for users to quickly select the appropriate expert when they need professional advice. Furthermore, the lack of means to accurately understand the user's intentions and provide appropriate responses during the consultation process resulted in a significant amount of time and effort being required. To address these issues, a more efficient and user-friendly consultation system is needed.

[0066] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0067] In this invention, the server includes means for generating an artificial intelligence program based on the consultation content entered by the user via an online infrastructure, means for storing user information using a secure communication protocol, and means for analyzing the entered consultation content using a natural language processing device to identify relevant topics and keywords. This enables the server to quickly understand the consultation content from the user, immediately select an appropriate expert, and conduct consultations in real time.

[0068] "Online infrastructure" refers to the entire platform that users access and use to send and receive information via the internet.

[0069] "Users" refers to individual users who use the system to input their consultation details and receive advice and information from experts.

[0070] An "artificial intelligence program" refers to a program that analyzes the content of inquiries entered by users and provides appropriate responses or selects experts.

[0071] A "collection of expert information" refers to a database that compiles information on experts in various fields, and experts are selected based on this database.

[0072] A "secure communication protocol" refers to a means of communication that ensures the safe transmission and reception of data over the internet, and generally utilizes encryption technology.

[0073] A "natural language processing device" is a device equipped with technology to analyze the content of a user's inquiry into natural language and understand its content.

[0074] This invention constructs a consultation system that utilizes an online infrastructure to allow users to obtain expert advice. The system consists of three main components: a server, a user's terminal, and an expert's device.

[0075] The server plays a central role in receiving user inquiries and generating artificial intelligence programs. First, the server stores user information in a database using a secure communication protocol (e.g., HTTPS). When a user inputs their inquiry from their terminal, the server analyzes the content using a natural language processing system (e.g., a general-purpose natural language processing engine) to identify relevant topics and keywords. Based on this, the server utilizes a generative AI model to generate an artificial intelligence program tailored to the user's inquiry.

[0076] This artificial intelligence program is designed to provide instant answers to questions entered by users through their devices. Users can initiate consultations and receive answers via devices such as smartphones and personal computers. If expert advice is needed, the generated AI program selects a relevant expert from a database of expert information and retrieves that information via a server.

[0077] For example, if a user wants to "consult about home renovations," the server will select an expert in architecture or design and provide the user with specific advice. The user can immediately see the agent's response on their device and, if they have any questions, can use additional prompts to obtain further information.

[0078] As an example of a prompt, entering a message in the format of "I would like to consult with the AI ​​agent about a mortgage. Please select the appropriate expert and provide the user with the necessary information" will allow the user to receive a professional response tailored to their needs.

[0079] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0080] Step 1:

[0081] Users access the system using their device and enter their consultation details and personal information. The entered data is sent to the server. Specifically, users enter their name, contact information, and the content of their consultation in text format, and then press the send button on their device.

[0082] Step 2:

[0083] The server stores data received from the user in a database using a secure communication protocol. Here, the server encrypts the user data to prevent unauthorized external access. As output, user information is securely stored.

[0084] Step 3:

[0085] The server analyzes the stored consultation content using a natural language processing (NLP) system. The input is the text of the consultation content submitted by the user, and the server uses natural language processing techniques to extract the main topics and keywords. This analysis identifies the category and urgency of the consultation content. The output is a collection of the analyzed topics and keywords.

[0086] Step 4:

[0087] The server generates an artificial intelligence program based on the analysis results. Here, a response process tailored to the user's inquiry is designed using the generated AI model. The output is an AI-based response script tailored to the user's inquiry.

[0088] Step 5:

[0089] The server searches a database of expert information and selects the expert best suited to the analyzed consultation content. The input is the topic and keywords obtained in step 3, and the server runs an algorithm based on the expert profiles in the database. The output is the profile information of the selected expert.

[0090] Step 6:

[0091] The server feeds information from selected experts back into the artificial intelligence program, providing real-time advice to the user. Inputs include the profile information of the selected expert and the user's consultation topic. The server sends a response to the user's terminal, which the user can view on the screen and obtain further necessary information using prompts. Outputs are specific advice and suggestions displayed on the user's terminal.

[0092] (Application Example 1)

[0093] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0094] The problem this invention aims to solve is to enable users who require specialized knowledge regarding financial transactions and settlements to receive prompt and appropriate expert advice. Conventional systems have been cumbersome and inefficient in selecting experts. Furthermore, there is the problem of difficulty in finding the optimal solution for individual cases by utilizing expert advice.

[0095] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0096] In this invention, the server includes means for generating an artificial intelligence agent based on the content of a request sent by a user via an online platform, means for selecting the most suitable expert from an expert dataset using the artificial intelligence agent, and additional means for efficiently matching users with experts who specialize in financial transactions and settlements, and for providing expert advice from the experts. This enables users to effectively receive advice from experts and make optimal choices regarding transactions and settlements.

[0097] An "online platform" is a system that allows users to access various services and information via the internet.

[0098] An "artificial intelligence agent" is an automated computer program that analyzes user input, selects appropriate experts based on that input, and facilitates information exchange between users and experts.

[0099] A "specialist dataset" is a database containing numerous expert profiles, each of which records the expert's skills and experience.

[0100] "Automated response" refers to an immediate reply generated by an artificial intelligence agent in response to a user's request, and its content is intended to help the user solve their problem.

[0101] "Knowledge of financial transactions and settlements" refers to the specialized information and understanding required when selecting, contracting for, and using financial services and products.

[0102] "Expert advice" refers to practical and useful advice given by someone with expertise and experience in a particular field.

[0103] The system for implementing this invention consists of three basic components: a server, a user terminal, and a specialist device.

[0104] The server is the central element of the system, responsible for database management, artificial intelligence model generation, and collaboration with experts. The server first receives input from the user and analyzes it using a natural language processing engine. Based on the analysis results, an artificial intelligence agent is then generated. This agent interacts with the user, providing information to select the most suitable expert from the expert dataset. The selected expert then provides the user with expert advice on financial transactions and settlements.

[0105] The user's device is used to connect to the internet when the user seeks advice. The user inputs their inquiry into the system via the device and receives responses in real time. Furthermore, they can easily access information from experts based on a dataset of experts directly on their device. For example, if a user is struggling to choose a financial product, they can input their question via their smartphone and receive immediate advice from an expert.

[0106] The artificial intelligence agent within this system is built on a generative AI model and can facilitate smooth collaboration with experts through prompt messages. For example, the prompt message, "User is asking about investment advice on mutual funds. Identify the best expert to provide recommendations and explain the options," is input into the generative AI model, and an appropriate expert is selected based on it.

[0107] The servers and terminals utilize common cloud infrastructure and natural language processing libraries (e.g., NLTK, spaCy) to analyze user input and efficiently match them with experts, thereby facilitating optimal choices regarding transactions and payments.

[0108] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0109] Step 1:

[0110] The user's device receives the user's inquiry as input and sends it to the server. The input data is in text format and concerns inquiries about specific financial transactions or settlements. This data is transferred to the server as is.

[0111] Step 2:

[0112] The server uses a natural language processing engine to analyze the received text data. This analysis extracts keywords and intents from the consultation content. As a result of this analysis, the server outputs the specific consultation category and related topics.

[0113] Step 3:

[0114] The server generates an artificial intelligence agent using a generative AI model based on the analysis results. This agent searches an expert dataset using the generated prompt sentences. The prompt sentences include phrases like "User is asking about investment advice on mutual funds. Identify the best expert to provide recommendations and explain the options." which are generated based on the analysis results.

[0115] Step 4:

[0116] The server selects the most suitable expert from the generated prompt text and dataset. Using information from the expert dataset as input, the AI ​​agent outputs the most appropriate expert. The selected expert possesses the skills and experience best suited to the user's consultation.

[0117] Step 5:

[0118] The server retrieves advice from selected experts in real time and sends it to the user's terminal. The retrieved advice is output as structured text and notified to the user's terminal.

[0119] Step 6:

[0120] The user's device receives the consultation results and displays them to the user. The displayed content consists of suggestions and advice from experts, which the user can use to decide on their next course of action.

[0121] Step 7:

[0122] The user's device collects feedback from the user and sends it to the server. This feedback includes information about the user's satisfaction level and usefulness of the consultation, and is sent to the server as input.

[0123] Step 8:

[0124] The server processes user feedback to improve the artificial intelligence agent. The feedback data is analyzed and evaluated as input, helping to fine-tune and train the AI ​​model. This will enable more accurate matching and advice in future consultations.

[0125] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0126] This invention proposes a system that connects users with experts via an online platform, incorporating an emotion engine that recognizes the user's emotions. The system consists of a server, a user's terminal, and an expert's device, and is designed to ensure effective communication.

[0127] Server operation

[0128] The server receives inquiries from users and generates an artificial intelligence agent that understands the content. The agent uses a natural language processing engine to analyze the presented inquiries as text. Furthermore, an emotion engine extracts emotional nuances from the user's text and voice and evaluates the user's emotional state. Based on this data, the server selects the most appropriate response that takes emotions into consideration.

[0129] Terminal operation

[0130] On the user's device, users can input their inquiries via voice or text. The emotion engine analyzes the input voice tone and word choices to determine the user's emotional state. User input is directly transmitted to the server, and the agent's response is displayed in real time. This response is tailored to the user's emotional state, providing more accurate and empathetic support.

[0131] Collaboration with experts

[0132] If a user's question requires specialized knowledge, the AI ​​agent consults a database of experts and selects the appropriate expert. The selected expert can then use emotional information collected by the emotion engine to provide detailed advice tailored to the user's situation. This information supports the expert's judgment and enables more effective interaction.

[0133] Specific example

[0134] For example, if a user inputs "I want to talk about stress at work," the emotion engine detects the level of emotional tension from the input and voice. Based on this information, the system responds to the user in a gentle tone that is sensitive to their emotions. If necessary, a psychological counseling specialist is selected to provide more specific support. In this way, this system, which utilizes the emotion engine, reduces stress and provides a sense of security to users.

[0135] This configuration allows the consultation process between users and professionals to address emotional aspects, thus enabling more comprehensive support.

[0136] The following describes the processing flow.

[0137] Step 1:

[0138] The user accesses the online platform using their device and logs into their account. The user then enters the details of their inquiry via text or voice.

[0139] Step 2:

[0140] The server receives user input in real time and activates a natural language processing engine. It analyzes the input content and identifies the category of the inquiry.

[0141] Step 3:

[0142] The emotion engine analyzes the user's input text and voice, and evaluates the user's emotional state based on the words used and their tone of voice.

[0143] Step 4:

[0144] The server generates an appropriate artificial intelligence agent based on the results of natural language processing and sentiment analysis. The agent prepares the optimal response according to the user's inquiry and emotional state.

[0145] Step 5:

[0146] The server sends a response to the user's terminal via an agent. The user can view suggestions and advice displayed on the screen in real time.

[0147] Step 6:

[0148] If the consultation requires further specialized knowledge, the server will refer to a database of experts and select an expert that best suits the user's needs.

[0149] Step 7:

[0150] The selected experts can take into account the information obtained from sentiment analysis to provide more accurate and emotionally resonant advice. The server delivers the experts' responses to the user via an artificial intelligence agent.

[0151] Step 8:

[0152] We have created an environment where users can optionally provide feedback after completing a consultation. The server collects this feedback and uses it to improve future artificial intelligence agents.

[0153] (Example 2)

[0154] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0155] Generating responses that align with users' emotional needs in online consultation support systems is challenging due to the complex and diverse technologies involved. Conventional technologies are insufficient in accurately understanding users' emotional states and responding appropriately to their concerns. There is a need for a system that builds deep trust through emotionally empathetic responses, thereby achieving more effective support.

[0156] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0157] In this invention, the server includes means for receiving consultation content input from a user via a communication network, means for generating an artificial intelligence agent using a natural language processing engine to analyze the consultation content, and means for evaluating the user's emotional state using an emotion analysis engine. This enables automated responses and expert selection adapted to the user.

[0158] A "communication network" is a system that enables the transmission and reception of data between multiple devices.

[0159] "Consultation content" refers to information about problems or questions that users need to resolve.

[0160] A "natural language processing engine" is software that processes text data using technology that analyzes and understands human language.

[0161] An "artificial intelligence agent" is a program built to automatically perform specific tasks and has the ability to analyze user input and provide appropriate responses.

[0162] A "sentiment analysis engine" is a tool that extracts emotional nuances from text and audio to evaluate the user's emotional state.

[0163] "Automated response" refers to an answer to a user's question generated by an artificial intelligence agent.

[0164] A "specialist database" is a collection of data that compiles information on experts in various fields, and is used to select the appropriate expert.

[0165] An "algorithm" refers to a defined set of calculations or processing steps for solving a specific problem.

[0166] This invention is an online consultation system primarily consisting of a server, a user's terminal, and a specialist's device. The system aims to generate appropriate responses to consultations while considering the user's emotional state, and to match them with a specialist as needed.

[0167] The server processes user inquiries received via the communication network. These inquiries are input as text or audio and are first analyzed by a natural language processing engine. For example, spaCy, a publicly available natural language processing library, is used for this analysis. Based on the information obtained through the analysis, the server extracts the user's problem.

[0168] In parallel, the server uses an emotion analysis engine to extract emotional nuances from text and audio data. This analysis engine utilizes existing analysis tools capable of finely evaluating emotional tone. Based on the results of this emotion analysis, the server uses a generative AI model to generate an automated response in a tone that is empathetic to the user. This generation process utilizes the prompt message, "Generate a response that provides appropriate advice in a gentle tone, based on the user's inquiry and emotional state."

[0169] If a user's problem requires advanced expertise, the server consults a database of experts and selects the most relevant one. This database contains profile information based on each expert's skills and achievements.

[0170] On the user's device, user input is sent to the server in real time, and the response from the server is displayed immediately. This allows users to receive prompt support.

[0171] For example, if a user inputs "I want to talk about stress at work," the emotion analysis engine detects the level of emotional tension from the input and voice. Based on this information, the system provides the user with a gentle and empathetic response, and if necessary, selects a psychological counseling specialist to provide detailed support.

[0172] Thus, this system aims to provide more accurate and effective online consultations while taking into account the user's feelings.

[0173] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0174] Step 1:

[0175] The user enters their inquiry details using a terminal. Input can be via voice or text. In the case of voice input, the terminal acquires the voice data and converts it to text using speech recognition software. At this time, either voice or text is obtained as input data. For voice input, data is acquired through the microphone, and the text data is output via speech recognition.

[0176] Step 2:

[0177] The terminal sends the acquired text data to the server. HTTPS, a secure communication protocol, is used for this process. The input data is in text format and is sent to the server as is.

[0178] Step 3:

[0179] The server analyzes the received text data using a natural language processing engine. Specifically, it uses a natural language processing library to analyze grammatical structure and extract keywords. The input data is text, and the output provides extracted keywords and sentence structure information.

[0180] Step 4:

[0181] The server passes the analyzed data to the sentiment analysis engine, which evaluates the user's emotions. The sentiment analysis engine extracts emotional nuances from the text data and quantifies the emotional state. The input is the text data of the analysis results, and the output is an emotion score.

[0182] Step 5:

[0183] The server generates a response using a generative AI model based on the emotion score and text data. This process creates a response with the prompt set to "Generate a response that provides appropriate advice in a gentle tone, based on the user's inquiry and emotional state." The input is the emotion score and text data, and the output is the generated response text.

[0184] Step 6:

[0185] If the consultation requires specialized knowledge, the server will refer to a database of experts and select the appropriate expert. A database search algorithm is used to identify highly relevant expert profiles. The input is keywords related to the consultation, and the output is information on the selected expert.

[0186] Step 7:

[0187] The server sends the generated response and expert information to the terminal. The data is encrypted using the HTTPS protocol, allowing the user to receive results in real time.

[0188] Step 8:

[0189] The terminal displays the response received from the server to the user. The displayed content is output in a format adapted to the user's screen and can be checked immediately. This allows the user to receive quick and emotionally sensitive support.

[0190] (Application Example 2)

[0191] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0192] This invention aims to effectively recognize the emotional state of users and generate emotionally sensitive responses in a consultation system via an online platform. Conventional systems select experts based on user input, but struggle to provide responses that take emotional nuances into account, making it difficult to provide empathetic support to users.

[0193] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0194] In this invention, the server includes means for generating an artificial intelligence agent based on the consultation content entered by the user via an online platform, means for evaluating the user's emotional state with a module equipped with an emotion recognition function, and means for generating an emotion-sensitive response based on the evaluated emotional state. This enables empathetic and effective consultation that takes the user's emotions into consideration.

[0195] An "online platform" is a system that users can access via the internet and utilize various services and functions.

[0196] "User" refers to an individual or group that uses a service or system.

[0197] "Consultation content" refers to information about questions or problems that users present to experts or the system.

[0198] An "artificial intelligence agent" is a program that simulates intelligent tasks like those performed by humans, processing data and making decisions.

[0199] The "emotion recognition function" is a function that analyzes the user's language expression and tone of voice to determine their emotional state.

[0200] A "specialist database" is a data bank that compiles information on experts with knowledge and experience in a specific field.

[0201] "Real-time" refers to a system that responds quickly to user actions and processes them immediately.

[0202] "Generating a response" means preparing answers or reactions that the system will generate based on user input.

[0203] "Collecting feedback" means gathering opinions and evaluations from users and using them as material for improvement and evaluation.

[0204] The system for realizing this invention provides consultation services incorporating emotion recognition capabilities via an online platform, and is configured as follows.

[0205] The server operates an emotion recognition system built using TENSORFLOW®. When a consultation request is entered, the server sends the voice and text data from the user to a cloud service. The transmitted data is analyzed using spaCy, a natural language processing engine, to evaluate the emotional state. Based on this evaluation, an optimal response that takes emotions into consideration is generated and provided to the user in real time.

[0206] The terminal receives user input and transfers the data to the server. The terminal has a built-in microphone, camera, and speaker, and uses this hardware to collect audio and image data during consultations.

[0207] As a concrete example, if a user asks a home robot, "How can I relieve stress at work?", the robot records the utterance as audio on the device and sends it to a server. The server analyzes the emotions and generates specific advice to alleviate stress.

[0208] An example of a prompt used by a generative AI model is, "Think of the best thing to say to a user who is feeling stressed." Based on this prompt, the system can generate a thoughtful response.

[0209] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0210] Step 1:

[0211] User input

[0212] The user inputs their inquiry into the terminal via voice or text. The terminal acquires the voice or text data using a microphone and text input device. This input data forms the basis for the next processing step.

[0213] Step 2:

[0214] Sending data

[0215] The terminal sends the acquired voice and text data to the server. The transmission is performed using a secure protocol to maintain real-time data availability.

[0216] Step 3:

[0217] Speech and text analysis

[0218] The server analyzes the received data. First, the audio data is converted to text using speech recognition software. Next, this text data is analyzed by spaCy, a natural language processing engine, to extract important keywords and context. This analysis provides the information necessary for subsequent sentiment evaluation.

[0219] Step 4:

[0220] Assessment of emotional state

[0221] The server uses a TensorFlow-based emotion recognition module to evaluate the analyzed text data. This evaluation infers the user's emotional state from the text's vocabulary and speech tone. For example, emotions such as stress and sadness are identified. Based on the evaluation results, an optimal response tailored to the emotion is generated.

[0222] Step 5:

[0223] Generating the optimal response

[0224] Based on the evaluation of the emotional state, the server uses a generative AI model to generate a response using a prompt (e.g., "Please think of the best thing to say when you are feeling stressed."). This response is configured to be sensitive to the user's emotions.

[0225] Step 6:

[0226] Response forwarding and display

[0227] The generated response is sent to the terminal. The terminal presents this to the user as voice or text. The terminal's speaker and display are utilized to provide real-time responses, allowing the user to receive personalized support.

[0228] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0229] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0230] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0231] [Second Embodiment]

[0232] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0233] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0234] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0235] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0236] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0237] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0238] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0239] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0240] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0241] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0242] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0243] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0244] As a means of implementing this invention, a system for consulting with experts using an online platform is constructed. The system consists of three main components: a server, a user's terminal, and an expert's device.

[0245] Server operation

[0246] The server plays a central role in this system, managing the database, generating artificial intelligence agents, and coordinating with experts. First, the server receives registered user information and stores it in the database. Then, based on the user's input, it uses a natural language processing engine to analyze the consultation content and generate an appropriate artificial intelligence agent. This agent facilitates smooth conversations with the user while utilizing information relevant to the consultation content to select the most suitable expert from the expert database.

[0247] Terminal operation

[0248] The user's device is an internet-connected device that, after registration, allows the user to consult with an artificial intelligence agent. Users can input consultation details related to their needs and receive responses in real time. If expert advice is needed, the AI ​​agent collects the information, queries the server, and provides appropriate real-time feedback.

[0249] Specific example

[0250] For example, if a user wants to "consult about a mortgage," the AI ​​agent understands the user's inquiry and sends that information to the server. The server searches a database of experts and selects an appropriate expert in the financial field. Based on the expert's registered profile, the AI ​​agent provides the user with expert advice and solutions. The user can easily check the agent's response on their device.

[0251] This specific example demonstrates that users can efficiently seek expert advice, while experts can create an efficient consultation environment that allows them to respond to a larger number of inquiries.

[0252] The following describes the processing flow.

[0253] Step 1:

[0254] The server verifies the user's access on the online platform and displays the account registration screen. The user enters their personal information and the area of ​​consultation they wish to pursue, and submits the registration form.

[0255] Step 2:

[0256] The server receives registration information submitted by the user, verifies the input, and then saves it to the database. After saving is complete, an automatic confirmation email is sent to the user to notify them of the registration completion.

[0257] Step 3:

[0258] The server invokes a natural language processing engine based on registered user information to generate an artificial intelligence agent that responds to the user's request. This agent includes an optimal language model based on the user's requirements.

[0259] Step 4:

[0260] The user logs into the system using their device and initiates a consultation through an artificial intelligence agent. The consultation details entered by the user are immediately analyzed by the agent.

[0261] Step 5:

[0262] The server searches a database of experts based on the consultation details received via the artificial intelligence agent. An algorithm then selects the expert best suited to the user's needs.

[0263] Step 6:

[0264] The server sends information on selected experts to an artificial intelligence agent, which then retrieves advice from the experts in real time and notifies the user.

[0265] Step 7:

[0266] The user reviews the advice provided by the agent on their device and asks further questions as needed. The agent continues to respond appropriately to these questions.

[0267] Step 8:

[0268] After the consultation ends, the server provides the user with a feedback format and stores the collected feedback in a database. This can then be used to improve future artificial intelligence agents.

[0269] (Example 1)

[0270] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0271] Conventional consultation systems have made it difficult for users to quickly select the appropriate expert when they need professional advice. Furthermore, the lack of means to accurately understand the user's intentions and provide appropriate responses during the consultation process resulted in a significant amount of time and effort being required. To address these issues, a more efficient and user-friendly consultation system is needed.

[0272] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0273] In this invention, the server includes means for generating an artificial intelligence program based on the consultation content entered by the user via an online infrastructure, means for storing user information using a secure communication protocol, and means for analyzing the entered consultation content using a natural language processing device to identify relevant topics and keywords. This enables the server to quickly understand the consultation content from the user, immediately select an appropriate expert, and conduct consultations in real time.

[0274] "Online infrastructure" refers to the entire platform that users access and use to send and receive information via the internet.

[0275] "Users" refers to individual users who use the system to input their consultation details and receive advice and information from experts.

[0276] An "artificial intelligence program" refers to a program that analyzes the content of inquiries entered by users and provides appropriate responses or selects experts.

[0277] A "collection of expert information" refers to a database that compiles information on experts in various fields, and experts are selected based on this database.

[0278] A "secure communication protocol" refers to a means of communication that ensures the safe transmission and reception of data over the internet, and generally utilizes encryption technology.

[0279] A "natural language processing device" is a device equipped with technology to analyze the content of a user's inquiry into natural language and understand its content.

[0280] This invention constructs a consultation system that utilizes an online infrastructure to allow users to obtain expert advice. The system consists of three main components: a server, a user's terminal, and an expert's device.

[0281] The server plays a central role in receiving user inquiries and generating artificial intelligence programs. First, the server stores user information in a database using a secure communication protocol (e.g., HTTPS). When a user inputs their inquiry from their terminal, the server analyzes the content using a natural language processing system (e.g., a general-purpose natural language processing engine) to identify relevant topics and keywords. Based on this, the server utilizes a generative AI model to generate an artificial intelligence program tailored to the user's inquiry.

[0282] This artificial intelligence program is designed to provide immediate answers to questions input by users through terminals. Users can start consultations through terminals such as smartphones and personal computers and receive answers. When advice from experts is needed, the generated artificial intelligence program selects relevant experts from the expert information collection and obtains that information via the server.

[0283] As a specific example, when a user wants to consult about home renovation, the server selects experts related to architecture and design and provides specific advice to the user. The user can immediately check the answer from the agent on the terminal and, if there are questions, obtain further information using additional prompts.

[0284] As an example of a prompt sentence, by inputting in the form of "I want to consult an AI agent about a home loan. Please select an appropriate expert and provide the necessary information to the user.", a professional response tailored to the user's needs can be obtained.

[0285] The flow of the specific process in Example 1 will be described using FIG. 11.

[0286] Step 1:

[0287] The user uses the terminal to access the system and inputs consultation content and personal information. The input data is sent to the server. Specifically, the user inputs their name, contact information, and the content they want to consult in text form and presses the send button on the terminal.

[0288] Step 2:

[0289] The server saves the data received from the user in the database using a secure communication protocol. Here, the server encrypts the user data and performs processing to prevent unauthorized access from the outside. As output, the user information is securely saved.

[0290] Step 3:

[0291] The server analyzes the stored consultation content using a natural language processing (NLP) system. The input is the text of the consultation content submitted by the user, and the server uses natural language processing techniques to extract the main topics and keywords. This analysis identifies the category and urgency of the consultation content. The output is a collection of the analyzed topics and keywords.

[0292] Step 4:

[0293] The server generates an artificial intelligence program based on the analysis results. Here, a response process tailored to the user's inquiry is designed using the generated AI model. The output is an AI-based response script tailored to the user's inquiry.

[0294] Step 5:

[0295] The server searches a database of expert information and selects the expert best suited to the analyzed consultation content. The input is the topic and keywords obtained in step 3, and the server runs an algorithm based on the expert profiles in the database. The output is the profile information of the selected expert.

[0296] Step 6:

[0297] The server feeds information from selected experts back into the artificial intelligence program, providing real-time advice to the user. Inputs include the profile information of the selected expert and the user's consultation topic. The server sends a response to the user's terminal, which the user can view on the screen and obtain further necessary information using prompts. Outputs are specific advice and suggestions displayed on the user's terminal.

[0298] (Application Example 1)

[0299] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0300] The problem this invention aims to solve is to enable users who require specialized knowledge regarding financial transactions and settlements to receive prompt and appropriate expert advice. Conventional systems have been cumbersome and inefficient in selecting experts. Furthermore, there is the problem of difficulty in finding the optimal solution for individual cases by utilizing expert advice.

[0301] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0302] In this invention, the server includes means for generating an artificial intelligence agent based on the content of a request sent by a user via an online platform, means for selecting the most suitable expert from an expert dataset using the artificial intelligence agent, and additional means for efficiently matching users with experts who specialize in financial transactions and settlements, and for providing expert advice from the experts. This enables users to effectively receive advice from experts and make optimal choices regarding transactions and settlements.

[0303] An "online platform" is a system that allows users to access various services and information via the internet.

[0304] An "artificial intelligence agent" is an automated computer program that analyzes user input, selects appropriate experts based on that input, and facilitates information exchange between users and experts.

[0305] A "specialist dataset" is a database containing numerous expert profiles, each of which records the expert's skills and experience.

[0306] "Automatic response" refers to an immediate response generated by an artificial intelligence agent to the content of a request from a user, and its content is to assist the user in solving problems.

[0307] "Knowledge related to financial transactions and settlements" refers to specialized information and understanding required in the selection, contract, and use of financial services and products.

[0308] "Professional advice" refers to practical and useful advice given by a person with specialized knowledge and experience in a specific field.

[0309] The system for implementing this invention consists of three basic components: a server, a user's terminal, and an expert's device.

[0310] The server is the central element of the system and is responsible for database management, generating artificial intelligence models, and collaborating with experts. The server first receives the input content from the user and analyzes the content using a natural language processing engine. Then, based on the analysis results, an artificial intelligence agent is generated. This agent provides information for selecting the most suitable expert from the expert dataset through interaction with the user. The selected expert provides professional advice on financial transactions and settlements to the user.

[0311] The user's terminal is a device for connecting to the Internet when the user consults. The user inputs the consultation content into the system through the terminal and receives a response in real time. Also, information from experts based on the expert dataset can be easily confirmed on the terminal. As a specific example, when a user is troubled about how to choose a financial product, the user can input the question through a smartphone and immediately obtain advice from an expert.

[0312] The artificial intelligence agent within this system is built on a generative AI model and can facilitate smooth collaboration with experts through prompt messages. For example, the prompt message, "User is asking about investment advice on mutual funds. Identify the best expert to provide recommendations and explain the options," is input into the generative AI model, and an appropriate expert is selected based on it.

[0313] The servers and terminals utilize common cloud infrastructure and natural language processing libraries (e.g., NLTK, spaCy) to analyze user input and efficiently match them with experts, thereby facilitating optimal choices regarding transactions and payments.

[0314] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0315] Step 1:

[0316] The user's device receives the user's inquiry as input and sends it to the server. The input data is in text format and concerns inquiries about specific financial transactions or settlements. This data is transferred to the server as is.

[0317] Step 2:

[0318] The server uses a natural language processing engine to analyze the received text data. This analysis extracts keywords and intents from the consultation content. As a result of this analysis, the server outputs the specific consultation category and related topics.

[0319] Step 3:

[0320] The server generates an artificial intelligence agent using a generative AI model based on the analysis results. This agent searches an expert dataset using the generated prompt sentences. The prompt sentences include phrases like "User is asking about investment advice on mutual funds. Identify the best expert to provide recommendations and explain the options." which are generated based on the analysis results.

[0321] Step 4:

[0322] The server selects the most suitable expert from the generated prompt text and dataset. Using information from the expert dataset as input, the AI ​​agent outputs the most appropriate expert. The selected expert possesses the skills and experience best suited to the user's consultation.

[0323] Step 5:

[0324] The server retrieves advice from selected experts in real time and sends it to the user's terminal. The retrieved advice is output as structured text and notified to the user's terminal.

[0325] Step 6:

[0326] The user's device receives the consultation results and displays them to the user. The displayed content consists of suggestions and advice from experts, which the user can use to decide on their next course of action.

[0327] Step 7:

[0328] The user's device collects feedback from the user and sends it to the server. This feedback includes information about the user's satisfaction level and usefulness of the consultation, and is sent to the server as input.

[0329] Step 8:

[0330] The server processes user feedback to improve the artificial intelligence agent. The feedback data is analyzed and evaluated as input, helping to fine-tune and train the AI ​​model. This will enable more accurate matching and advice in future consultations.

[0331] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0332] This invention proposes a system that connects users with experts via an online platform, incorporating an emotion engine that recognizes the user's emotions. The system consists of a server, a user's terminal, and an expert's device, and is designed to ensure effective communication.

[0333] Server operation

[0334] The server receives inquiries from users and generates an artificial intelligence agent that understands the content. The agent uses a natural language processing engine to analyze the presented inquiries as text. Furthermore, an emotion engine extracts emotional nuances from the user's text and voice and evaluates the user's emotional state. Based on this data, the server selects the most appropriate response that takes emotions into consideration.

[0335] Terminal operation

[0336] On the user's device, users can input their inquiries via voice or text. The emotion engine analyzes the input voice tone and word choices to determine the user's emotional state. User input is directly transmitted to the server, and the agent's response is displayed in real time. This response is tailored to the user's emotional state, providing more accurate and empathetic support.

[0337] Collaboration with experts

[0338] If a user's question requires specialized knowledge, the AI ​​agent consults a database of experts and selects the appropriate expert. The selected expert can then use emotional information collected by the emotion engine to provide detailed advice tailored to the user's situation. This information supports the expert's judgment and enables more effective interaction.

[0339] Specific example

[0340] For example, if a user inputs "I want to talk about stress at work," the emotion engine detects the level of emotional tension from the input and voice. Based on this information, the system responds to the user in a gentle tone that is sensitive to their emotions. If necessary, a psychological counseling specialist is selected to provide more specific support. In this way, this system, which utilizes the emotion engine, reduces stress and provides a sense of security to users.

[0341] This configuration allows the consultation process between users and professionals to address emotional aspects, thus enabling more comprehensive support.

[0342] The following describes the processing flow.

[0343] Step 1:

[0344] The user accesses the online platform using their device and logs into their account. The user then enters the details of their inquiry via text or voice.

[0345] Step 2:

[0346] The server receives user input in real time and activates a natural language processing engine. It analyzes the input content and identifies the category of the inquiry.

[0347] Step 3:

[0348] The emotion engine analyzes the user's input text and voice, and evaluates the user's emotional state based on the words used and their tone of voice.

[0349] Step 4:

[0350] The server generates an appropriate artificial intelligence agent based on the results of natural language processing and sentiment analysis. The agent prepares the optimal response according to the user's inquiry and emotional state.

[0351] Step 5:

[0352] The server sends a response to the user's terminal via an agent. The user can view suggestions and advice displayed on the screen in real time.

[0353] Step 6:

[0354] If the consultation requires further specialized knowledge, the server will refer to a database of experts and select an expert that best suits the user's needs.

[0355] Step 7:

[0356] The selected experts can take into account the information obtained from sentiment analysis to provide more accurate and emotionally resonant advice. The server delivers the experts' responses to the user via an artificial intelligence agent.

[0357] Step 8:

[0358] We have created an environment where users can optionally provide feedback after completing a consultation. The server collects this feedback and uses it to improve future artificial intelligence agents.

[0359] (Example 2)

[0360] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0361] Generating responses that align with users' emotional needs in online consultation support systems is challenging due to the complex and diverse technologies involved. Conventional technologies are insufficient in accurately understanding users' emotional states and responding appropriately to their concerns. There is a need for a system that builds deep trust through emotionally empathetic responses, thereby achieving more effective support.

[0362] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0363] In this invention, the server includes means for receiving consultation content input from a user via a communication network, means for generating an artificial intelligence agent using a natural language processing engine to analyze the consultation content, and means for evaluating the user's emotional state using an emotion analysis engine. This enables automated responses and expert selection adapted to the user.

[0364] A "communication network" is a system that enables the transmission and reception of data between multiple devices.

[0365] "Consultation content" refers to information about problems or questions that users need to resolve.

[0366] A "natural language processing engine" is software that processes text data using technology that analyzes and understands human language.

[0367] An "artificial intelligence agent" is a program built to automatically perform specific tasks and has the ability to analyze user input and provide appropriate responses.

[0368] A "sentiment analysis engine" is a tool that extracts emotional nuances from text and audio to evaluate the user's emotional state.

[0369] "Automated response" refers to an answer to a user's question generated by an artificial intelligence agent.

[0370] A "specialist database" is a collection of data that compiles information on experts in various fields, and is used to select the appropriate expert.

[0371] An "algorithm" refers to a defined set of calculations or processing steps for solving a specific problem.

[0372] This invention is an online consultation system primarily consisting of a server, a user's terminal, and a specialist's device. The system aims to generate appropriate responses to consultations while considering the user's emotional state, and to match them with a specialist as needed.

[0373] The server processes user inquiries received via the communication network. These inquiries are input as text or audio and are first analyzed by a natural language processing engine. For example, spaCy, a publicly available natural language processing library, is used for this analysis. Based on the information obtained through the analysis, the server extracts the user's problem.

[0374] In parallel, the server uses an emotion analysis engine to extract emotional nuances from text and audio data. This analysis engine utilizes existing analysis tools capable of finely evaluating emotional tone. Based on the results of this emotion analysis, the server uses a generative AI model to generate an automated response in a tone that is empathetic to the user. This generation process utilizes the prompt message, "Generate a response that provides appropriate advice in a gentle tone, based on the user's inquiry and emotional state."

[0375] If a user's problem requires advanced expertise, the server consults a database of experts and selects the most relevant one. This database contains profile information based on each expert's skills and achievements.

[0376] On the user's device, user input is sent to the server in real time, and the response from the server is displayed immediately. This allows users to receive prompt support.

[0377] For example, if a user inputs "I want to talk about stress at work," the emotion analysis engine detects the level of emotional tension from the input and voice. Based on this information, the system provides the user with a gentle and empathetic response, and if necessary, selects a psychological counseling specialist to provide detailed support.

[0378] Thus, this system aims to provide more accurate and effective online consultations while taking into account the user's feelings.

[0379] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0380] Step 1:

[0381] The user enters their inquiry details using a terminal. Input can be via voice or text. In the case of voice input, the terminal acquires the voice data and converts it to text using speech recognition software. At this time, either voice or text is obtained as input data. For voice input, data is acquired through the microphone, and the text data is output via speech recognition.

[0382] Step 2:

[0383] The terminal sends the acquired text data to the server. HTTPS, a secure communication protocol, is used for this process. The input data is in text format and is sent to the server as is.

[0384] Step 3:

[0385] The server analyzes the received text data using a natural language processing engine. Specifically, it uses a natural language processing library to analyze grammatical structure and extract keywords. The input data is text, and the output provides extracted keywords and sentence structure information.

[0386] Step 4:

[0387] The server passes the analyzed data to the sentiment analysis engine, which evaluates the user's emotions. The sentiment analysis engine extracts emotional nuances from the text data and quantifies the emotional state. The input is the text data of the analysis results, and the output is an emotion score.

[0388] Step 5:

[0389] The server generates a response using a generative AI model based on the emotion score and text data. This process creates a response with the prompt set to "Generate a response that provides appropriate advice in a gentle tone, based on the user's inquiry and emotional state." The input is the emotion score and text data, and the output is the generated response text.

[0390] Step 6:

[0391] If the consultation requires specialized knowledge, the server will refer to a database of experts and select the appropriate expert. A database search algorithm is used to identify highly relevant expert profiles. The input is keywords related to the consultation, and the output is information on the selected expert.

[0392] Step 7:

[0393] The server sends the generated response and expert information to the terminal. The data is encrypted using the HTTPS protocol, allowing the user to receive results in real time.

[0394] Step 8:

[0395] The terminal displays the response received from the server to the user. The displayed content is output in a format adapted to the user's screen and can be checked immediately. This allows the user to receive quick and emotionally sensitive support.

[0396] (Application Example 2)

[0397] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0398] This invention aims to effectively recognize the emotional state of users and generate emotionally sensitive responses in a consultation system via an online platform. Conventional systems select experts based on user input, but struggle to provide responses that take emotional nuances into account, making it difficult to provide empathetic support to users.

[0399] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0400] In this invention, the server includes means for generating an artificial intelligence agent based on the consultation content entered by the user via an online platform, means for evaluating the user's emotional state with a module equipped with an emotion recognition function, and means for generating an emotion-sensitive response based on the evaluated emotional state. This enables empathetic and effective consultation that takes the user's emotions into consideration.

[0401] An "online platform" is a system that users can access via the internet and utilize various services and functions.

[0402] "User" refers to an individual or group that uses a service or system.

[0403] "Consultation content" refers to information about questions or problems that users present to experts or the system.

[0404] An "artificial intelligence agent" is a program that simulates intelligent tasks like those performed by humans, processing data and making decisions.

[0405] The "emotion recognition function" is a function that analyzes the user's language expression and tone of voice to determine their emotional state.

[0406] A "specialist database" is a data bank that compiles information on experts with knowledge and experience in a specific field.

[0407] "Real-time" refers to a system that responds quickly to user actions and processes them immediately.

[0408] "Generating a response" means preparing answers or reactions that the system will generate based on user input.

[0409] "Collecting feedback" means gathering opinions and evaluations from users and using them as material for improvement and evaluation.

[0410] The system for realizing this invention provides consultation services incorporating emotion recognition capabilities via an online platform, and is configured as follows.

[0411] The server runs an emotion recognition system built using TensorFlow. When a consultation request is entered, the server sends the voice and text data from the user to a cloud service. The transmitted data is analyzed using spaCy, a natural language processing engine, to evaluate the emotional state. Based on this evaluation, an optimal response that takes emotions into consideration is generated and provided to the user in real time.

[0412] The terminal receives user input and transfers the data to the server. The terminal has a built-in microphone, camera, and speaker, and uses this hardware to collect audio and image data during consultations.

[0413] As a concrete example, if a user asks a home robot, "How can I relieve stress at work?", the robot records the utterance as audio on the device and sends it to a server. The server analyzes the emotions and generates specific advice to alleviate stress.

[0414] An example of a prompt used by a generative AI model is, "Think of the best thing to say to a user who is feeling stressed." Based on this prompt, the system can generate a thoughtful response.

[0415] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0416] Step 1:

[0417] User input

[0418] The user inputs their inquiry into the terminal via voice or text. The terminal acquires the voice or text data using a microphone and text input device. This input data forms the basis for the next processing step.

[0419] Step 2:

[0420] Sending data

[0421] The terminal sends the acquired voice and text data to the server. The transmission is performed using a secure protocol to maintain real-time data availability.

[0422] Step 3:

[0423] Speech and text analysis

[0424] The server analyzes the received data. First, the audio data is converted to text using speech recognition software. Next, this text data is analyzed by spaCy, a natural language processing engine, to extract important keywords and context. This analysis provides the information necessary for subsequent sentiment evaluation.

[0425] Step 4:

[0426] Assessment of emotional state

[0427] The server uses a TensorFlow-based emotion recognition module to evaluate the analyzed text data. This evaluation infers the user's emotional state from the text's vocabulary and speech tone. For example, emotions such as stress and sadness are identified. Based on the evaluation results, an optimal response tailored to the emotion is generated.

[0428] Step 5:

[0429] Generating the optimal response

[0430] Based on the evaluation of the emotional state, the server uses a generative AI model to generate a response using a prompt (e.g., "Please think of the best thing to say when you are feeling stressed."). This response is configured to be sensitive to the user's emotions.

[0431] Step 6:

[0432] Response forwarding and display

[0433] The generated response is sent to the terminal. The terminal presents this to the user as voice or text. The terminal's speaker and display are utilized to provide real-time responses, allowing the user to receive personalized support.

[0434] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0435] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0436] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0437] [Third Embodiment]

[0438] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0439] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0440] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0441] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0442] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0443] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0444] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0445] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0446] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0447] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0448] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0449] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0450] As a means of implementing this invention, a system for consulting with experts using an online platform is constructed. The system consists of three main components: a server, a user's terminal, and an expert's device.

[0451] Server operation

[0452] The server plays a central role in this system, managing the database, generating artificial intelligence agents, and coordinating with experts. First, the server receives registered user information and stores it in the database. Then, based on the user's input, it uses a natural language processing engine to analyze the consultation content and generate an appropriate artificial intelligence agent. This agent facilitates smooth conversations with the user while utilizing information relevant to the consultation content to select the most suitable expert from the expert database.

[0453] Terminal operation

[0454] The user's device is an internet-connected device that, after registration, allows the user to consult with an artificial intelligence agent. Users can input consultation details related to their needs and receive responses in real time. If expert advice is needed, the AI ​​agent collects the information, queries the server, and provides appropriate real-time feedback.

[0455] Specific example

[0456] For example, if a user wants to "consult about a mortgage," the AI ​​agent understands the user's inquiry and sends that information to the server. The server searches a database of experts and selects an appropriate expert in the financial field. Based on the expert's registered profile, the AI ​​agent provides the user with expert advice and solutions. The user can easily check the agent's response on their device.

[0457] This specific example demonstrates that users can efficiently seek expert advice, while experts can create an efficient consultation environment that allows them to respond to a larger number of inquiries.

[0458] The following describes the processing flow.

[0459] Step 1:

[0460] The server verifies the user's access on the online platform and displays the account registration screen. The user enters their personal information and the area of ​​consultation they wish to pursue, and submits the registration form.

[0461] Step 2:

[0462] The server receives registration information submitted by the user, verifies the input, and then saves it to the database. After saving is complete, an automatic confirmation email is sent to the user to notify them of the registration completion.

[0463] Step 3:

[0464] The server invokes a natural language processing engine based on registered user information to generate an artificial intelligence agent that responds to the user's request. This agent includes an optimal language model based on the user's requirements.

[0465] Step 4:

[0466] The user logs into the system using their device and initiates a consultation through an artificial intelligence agent. The consultation details entered by the user are immediately analyzed by the agent.

[0467] Step 5:

[0468] The server searches a database of experts based on the consultation details received via the artificial intelligence agent. An algorithm then selects the expert best suited to the user's needs.

[0469] Step 6:

[0470] The server sends information on selected experts to an artificial intelligence agent, which then retrieves advice from the experts in real time and notifies the user.

[0471] Step 7:

[0472] The user reviews the advice provided by the agent on their device and asks further questions as needed. The agent continues to respond appropriately to these questions.

[0473] Step 8:

[0474] After the consultation ends, the server provides the user with a feedback format and stores the collected feedback in a database. This can then be used to improve future artificial intelligence agents.

[0475] (Example 1)

[0476] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0477] Conventional consultation systems have made it difficult for users to quickly select the appropriate expert when they need professional advice. Furthermore, the lack of means to accurately understand the user's intentions and provide appropriate responses during the consultation process resulted in a significant amount of time and effort being required. To address these issues, a more efficient and user-friendly consultation system is needed.

[0478] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0479] In this invention, the server includes means for generating an artificial intelligence program based on the consultation content entered by the user via an online infrastructure, means for storing user information using a secure communication protocol, and means for analyzing the entered consultation content using a natural language processing device to identify relevant topics and keywords. This enables the server to quickly understand the consultation content from the user, immediately select an appropriate expert, and conduct consultations in real time.

[0480] "Online infrastructure" refers to the entire platform that users access and use to send and receive information via the internet.

[0481] "Users" refers to individual users who use the system to input their consultation details and receive advice and information from experts.

[0482] An "artificial intelligence program" refers to a program that analyzes the content of inquiries entered by users and provides appropriate responses or selects experts.

[0483] A "collection of expert information" refers to a database that compiles information on experts in various fields, and experts are selected based on this database.

[0484] A "secure communication protocol" refers to a means of communication that ensures the safe transmission and reception of data over the internet, and generally utilizes encryption technology.

[0485] A "natural language processing device" is a device equipped with technology to analyze the content of a user's inquiry into natural language and understand its content.

[0486] This invention constructs a consultation system that utilizes an online infrastructure to allow users to obtain expert advice. The system consists of three main components: a server, a user's terminal, and an expert's device.

[0487] The server plays a central role in receiving user inquiries and generating artificial intelligence programs. First, the server stores user information in a database using a secure communication protocol (e.g., HTTPS). When a user inputs their inquiry from their terminal, the server analyzes the content using a natural language processing system (e.g., a general-purpose natural language processing engine) to identify relevant topics and keywords. Based on this, the server utilizes a generative AI model to generate an artificial intelligence program tailored to the user's inquiry.

[0488] This artificial intelligence program is designed to provide instant answers to questions entered by users through their devices. Users can initiate consultations and receive answers via devices such as smartphones and personal computers. If expert advice is needed, the generated AI program selects a relevant expert from a database of expert information and retrieves that information via a server.

[0489] For example, if a user wants to "consult about home renovations," the server will select an expert in architecture or design and provide the user with specific advice. The user can immediately see the agent's response on their device and, if they have any questions, can use additional prompts to obtain further information.

[0490] As an example of a prompt, entering a message in the format of "I would like to consult with the AI ​​agent about a mortgage. Please select the appropriate expert and provide the user with the necessary information" will allow the user to receive a professional response tailored to their needs.

[0491] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0492] Step 1:

[0493] Users access the system using their device and enter their consultation details and personal information. The entered data is sent to the server. Specifically, users enter their name, contact information, and the content of their consultation in text format, and then press the send button on their device.

[0494] Step 2:

[0495] The server stores data received from the user in a database using a secure communication protocol. Here, the server encrypts the user data to prevent unauthorized external access. As output, user information is securely stored.

[0496] Step 3:

[0497] The server analyzes the stored consultation content using a natural language processing (NLP) system. The input is the text of the consultation content submitted by the user, and the server uses natural language processing techniques to extract the main topics and keywords. This analysis identifies the category and urgency of the consultation content. The output is a collection of the analyzed topics and keywords.

[0498] Step 4:

[0499] The server generates an artificial intelligence program based on the analysis results. Here, a response process tailored to the user's inquiry is designed using the generated AI model. The output is an AI-based response script tailored to the user's inquiry.

[0500] Step 5:

[0501] The server searches a database of expert information and selects the expert best suited to the analyzed consultation content. The input is the topic and keywords obtained in step 3, and the server runs an algorithm based on the expert profiles in the database. The output is the profile information of the selected expert.

[0502] Step 6:

[0503] The server feeds information from selected experts back into the artificial intelligence program, providing real-time advice to the user. Inputs include the profile information of the selected expert and the user's consultation topic. The server sends a response to the user's terminal, which the user can view on the screen and obtain further necessary information using prompts. Outputs are specific advice and suggestions displayed on the user's terminal.

[0504] (Application Example 1)

[0505] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0506] The problem this invention aims to solve is to enable users who require specialized knowledge regarding financial transactions and settlements to receive prompt and appropriate expert advice. Conventional systems have been cumbersome and inefficient in selecting experts. Furthermore, there is the problem of difficulty in finding the optimal solution for individual cases by utilizing expert advice.

[0507] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0508] In this invention, the server includes means for generating an artificial intelligence agent based on the content of a request sent by a user via an online platform, means for selecting the most suitable expert from an expert dataset using the artificial intelligence agent, and additional means for efficiently matching users with experts who specialize in financial transactions and settlements, and for providing expert advice from the experts. This enables users to effectively receive advice from experts and make optimal choices regarding transactions and settlements.

[0509] An "online platform" is a system that allows users to access various services and information via the internet.

[0510] An "artificial intelligence agent" is an automated computer program that analyzes user input, selects appropriate experts based on that input, and facilitates information exchange between users and experts.

[0511] A "specialist dataset" is a database containing numerous expert profiles, each of which records the expert's skills and experience.

[0512] "Automated response" refers to an immediate reply generated by an artificial intelligence agent in response to a user's request, and its content is intended to help the user solve their problem.

[0513] "Knowledge of financial transactions and settlements" refers to the specialized information and understanding required when selecting, contracting for, and using financial services and products.

[0514] "Expert advice" refers to practical and useful advice given by someone with expertise and experience in a particular field.

[0515] The system for implementing this invention consists of three basic components: a server, a user terminal, and a specialist device.

[0516] The server is the central element of the system, responsible for database management, artificial intelligence model generation, and collaboration with experts. The server first receives input from the user and analyzes it using a natural language processing engine. Based on the analysis results, an artificial intelligence agent is then generated. This agent interacts with the user, providing information to select the most suitable expert from the expert dataset. The selected expert then provides the user with expert advice on financial transactions and settlements.

[0517] The user's device is used to connect to the internet when the user seeks advice. The user inputs their inquiry into the system via the device and receives responses in real time. Furthermore, they can easily access information from experts based on a dataset of experts directly on their device. For example, if a user is struggling to choose a financial product, they can input their question via their smartphone and receive immediate advice from an expert.

[0518] The artificial intelligence agent within this system is built on a generative AI model and can facilitate smooth collaboration with experts through prompt messages. For example, the prompt message, "User is asking about investment advice on mutual funds. Identify the best expert to provide recommendations and explain the options," is input into the generative AI model, and an appropriate expert is selected based on it.

[0519] The servers and terminals utilize common cloud infrastructure and natural language processing libraries (e.g., NLTK, spaCy) to analyze user input and efficiently match them with experts, thereby facilitating optimal choices regarding transactions and payments.

[0520] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0521] Step 1:

[0522] The user's device receives the user's inquiry as input and sends it to the server. The input data is in text format and concerns inquiries about specific financial transactions or settlements. This data is transferred to the server as is.

[0523] Step 2:

[0524] The server uses a natural language processing engine to analyze the received text data. This analysis extracts keywords and intents from the consultation content. As a result of this analysis, the server outputs the specific consultation category and related topics.

[0525] Step 3:

[0526] The server generates an artificial intelligence agent using a generative AI model based on the analysis results. This agent searches an expert dataset using the generated prompt sentences. The prompt sentences include phrases like "User is asking about investment advice on mutual funds. Identify the best expert to provide recommendations and explain the options." which are generated based on the analysis results.

[0527] Step 4:

[0528] The server selects the most suitable expert from the generated prompt text and dataset. Using information from the expert dataset as input, the AI ​​agent outputs the most appropriate expert. The selected expert possesses the skills and experience best suited to the user's consultation.

[0529] Step 5:

[0530] The server retrieves advice from selected experts in real time and sends it to the user's terminal. The retrieved advice is output as structured text and notified to the user's terminal.

[0531] Step 6:

[0532] The user's device receives the consultation results and displays them to the user. The displayed content consists of suggestions and advice from experts, which the user can use to decide on their next course of action.

[0533] Step 7:

[0534] The user's device collects feedback from the user and sends it to the server. This feedback includes information about the user's satisfaction level and usefulness of the consultation, and is sent to the server as input.

[0535] Step 8:

[0536] The server processes user feedback to improve the artificial intelligence agent. The feedback data is analyzed and evaluated as input, helping to fine-tune and train the AI ​​model. This will enable more accurate matching and advice in future consultations.

[0537] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0538] This invention proposes a system that connects users with experts via an online platform, incorporating an emotion engine that recognizes the user's emotions. The system consists of a server, a user's terminal, and an expert's device, and is designed to ensure effective communication.

[0539] Server operation

[0540] The server receives inquiries from users and generates an artificial intelligence agent that understands the content. The agent uses a natural language processing engine to analyze the presented inquiries as text. Furthermore, an emotion engine extracts emotional nuances from the user's text and voice and evaluates the user's emotional state. Based on this data, the server selects the most appropriate response that takes emotions into consideration.

[0541] Terminal operation

[0542] On the user's device, users can input their inquiries via voice or text. The emotion engine analyzes the input voice tone and word choices to determine the user's emotional state. User input is directly transmitted to the server, and the agent's response is displayed in real time. This response is tailored to the user's emotional state, providing more accurate and empathetic support.

[0543] Collaboration with experts

[0544] If a user's question requires specialized knowledge, the AI ​​agent consults a database of experts and selects the appropriate expert. The selected expert can then use emotional information collected by the emotion engine to provide detailed advice tailored to the user's situation. This information supports the expert's judgment and enables more effective interaction.

[0545] Specific example

[0546] For example, if a user inputs "I want to talk about stress at work," the emotion engine detects the level of emotional tension from the input and voice. Based on this information, the system responds to the user in a gentle tone that is sensitive to their emotions. If necessary, a psychological counseling specialist is selected to provide more specific support. In this way, this system, which utilizes the emotion engine, reduces stress and provides a sense of security to users.

[0547] This configuration allows the consultation process between users and professionals to address emotional aspects, thus enabling more comprehensive support.

[0548] The following describes the processing flow.

[0549] Step 1:

[0550] The user accesses the online platform using their device and logs into their account. The user then enters the details of their inquiry via text or voice.

[0551] Step 2:

[0552] The server receives user input in real time and activates a natural language processing engine. It analyzes the input content and identifies the category of the inquiry.

[0553] Step 3:

[0554] The emotion engine analyzes the user's input text and voice, and evaluates the user's emotional state based on the words used and their tone of voice.

[0555] Step 4:

[0556] The server generates an appropriate artificial intelligence agent based on the results of natural language processing and sentiment analysis. The agent prepares the optimal response according to the user's inquiry and emotional state.

[0557] Step 5:

[0558] The server sends a response to the user's terminal via an agent. The user can view suggestions and advice displayed on the screen in real time.

[0559] Step 6:

[0560] If the consultation requires further specialized knowledge, the server will refer to a database of experts and select an expert that best suits the user's needs.

[0561] Step 7:

[0562] The selected experts can take into account the information obtained from sentiment analysis to provide more accurate and emotionally resonant advice. The server delivers the experts' responses to the user via an artificial intelligence agent.

[0563] Step 8:

[0564] We have created an environment where users can optionally provide feedback after completing a consultation. The server collects this feedback and uses it to improve future artificial intelligence agents.

[0565] (Example 2)

[0566] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0567] Generating responses that align with users' emotional needs in online consultation support systems is challenging due to the complex and diverse technologies involved. Conventional technologies are insufficient in accurately understanding users' emotional states and responding appropriately to their concerns. There is a need for a system that builds deep trust through emotionally empathetic responses, thereby achieving more effective support.

[0568] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0569] In this invention, the server includes means for receiving consultation content input from a user via a communication network, means for generating an artificial intelligence agent using a natural language processing engine to analyze the consultation content, and means for evaluating the user's emotional state using an emotion analysis engine. This enables automated responses and expert selection adapted to the user.

[0570] A "communication network" is a system that enables the transmission and reception of data between multiple devices.

[0571] "Consultation content" refers to information about problems or questions that users need to resolve.

[0572] A "natural language processing engine" is software that processes text data using technology that analyzes and understands human language.

[0573] An "artificial intelligence agent" is a program built to automatically perform specific tasks and has the ability to analyze user input and provide appropriate responses.

[0574] A "sentiment analysis engine" is a tool that extracts emotional nuances from text and audio to evaluate the user's emotional state.

[0575] "Automated response" refers to an answer to a user's question generated by an artificial intelligence agent.

[0576] A "specialist database" is a collection of data that compiles information on experts in various fields, and is used to select the appropriate expert.

[0577] An "algorithm" refers to a defined set of calculations or processing steps for solving a specific problem.

[0578] This invention is an online consultation system primarily consisting of a server, a user's terminal, and a specialist's device. The system aims to generate appropriate responses to consultations while considering the user's emotional state, and to match them with a specialist as needed.

[0579] The server processes user inquiries received via the communication network. These inquiries are input as text or audio and are first analyzed by a natural language processing engine. For example, spaCy, a publicly available natural language processing library, is used for this analysis. Based on the information obtained through the analysis, the server extracts the user's problem.

[0580] In parallel, the server uses an emotion analysis engine to extract emotional nuances from text and audio data. This analysis engine utilizes existing analysis tools capable of finely evaluating emotional tone. Based on the results of this emotion analysis, the server uses a generative AI model to generate an automated response in a tone that is empathetic to the user. This generation process utilizes the prompt message, "Generate a response that provides appropriate advice in a gentle tone, based on the user's inquiry and emotional state."

[0581] If a user's problem requires advanced expertise, the server consults a database of experts and selects the most relevant one. This database contains profile information based on each expert's skills and achievements.

[0582] On the user's device, user input is sent to the server in real time, and the response from the server is displayed immediately. This allows users to receive prompt support.

[0583] For example, if a user inputs "I want to talk about stress at work," the emotion analysis engine detects the level of emotional tension from the input and voice. Based on this information, the system provides the user with a gentle and empathetic response, and if necessary, selects a psychological counseling specialist to provide detailed support.

[0584] Thus, this system aims to provide more accurate and effective online consultations while taking into account the user's feelings.

[0585] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0586] Step 1:

[0587] The user enters their inquiry details using a terminal. Input can be via voice or text. In the case of voice input, the terminal acquires the voice data and converts it to text using speech recognition software. At this time, either voice or text is obtained as input data. For voice input, data is acquired through the microphone, and the text data is output via speech recognition.

[0588] Step 2:

[0589] The terminal sends the acquired text data to the server. HTTPS, a secure communication protocol, is used for this process. The input data is in text format and is sent to the server as is.

[0590] Step 3:

[0591] The server analyzes the received text data using a natural language processing engine. Specifically, it uses a natural language processing library to analyze grammatical structure and extract keywords. The input data is text, and the output provides extracted keywords and sentence structure information.

[0592] Step 4:

[0593] The server passes the analyzed data to the sentiment analysis engine, which evaluates the user's emotions. The sentiment analysis engine extracts emotional nuances from the text data and quantifies the emotional state. The input is the text data of the analysis results, and the output is an emotion score.

[0594] Step 5:

[0595] The server generates a response using a generative AI model based on the emotion score and text data. This process creates a response with the prompt set to "Generate a response that provides appropriate advice in a gentle tone, based on the user's inquiry and emotional state." The input is the emotion score and text data, and the output is the generated response text.

[0596] Step 6:

[0597] If the consultation requires specialized knowledge, the server will refer to a database of experts and select the appropriate expert. A database search algorithm is used to identify highly relevant expert profiles. The input is keywords related to the consultation, and the output is information on the selected expert.

[0598] Step 7:

[0599] The server sends the generated response and expert information to the terminal. The data is encrypted using the HTTPS protocol, allowing the user to receive results in real time.

[0600] Step 8:

[0601] The terminal displays the response received from the server to the user. The displayed content is output in a format adapted to the user's screen and can be checked immediately. This allows the user to receive quick and emotionally sensitive support.

[0602] (Application Example 2)

[0603] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0604] This invention aims to effectively recognize the emotional state of users and generate emotionally sensitive responses in a consultation system via an online platform. Conventional systems select experts based on user input, but struggle to provide responses that take emotional nuances into account, making it difficult to provide empathetic support to users.

[0605] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0606] In this invention, the server includes means for generating an artificial intelligence agent based on the consultation content entered by the user via an online platform, means for evaluating the user's emotional state with a module equipped with an emotion recognition function, and means for generating an emotion-sensitive response based on the evaluated emotional state. This enables empathetic and effective consultation that takes the user's emotions into consideration.

[0607] An "online platform" is a system that users can access via the internet and utilize various services and functions.

[0608] "User" refers to an individual or group that uses a service or system.

[0609] "Consultation content" refers to information about questions or problems that users present to experts or the system.

[0610] An "artificial intelligence agent" is a program that simulates intelligent tasks like those performed by humans, processing data and making decisions.

[0611] The "emotion recognition function" is a function that analyzes the user's language expression and tone of voice to determine their emotional state.

[0612] A "specialist database" is a data bank that compiles information on experts with knowledge and experience in a specific field.

[0613] "Real-time" refers to a system that responds quickly to user actions and processes them immediately.

[0614] "Generating a response" means preparing answers or reactions that the system will generate based on user input.

[0615] "Collecting feedback" means gathering opinions and evaluations from users and using them as material for improvement and evaluation.

[0616] The system for realizing this invention provides consultation services incorporating emotion recognition capabilities via an online platform, and is configured as follows.

[0617] The server runs an emotion recognition system built using TensorFlow. When a consultation request is entered, the server sends the voice and text data from the user to a cloud service. The transmitted data is analyzed using spaCy, a natural language processing engine, to evaluate the emotional state. Based on this evaluation, an optimal response that takes emotions into consideration is generated and provided to the user in real time.

[0618] The terminal receives user input and transfers the data to the server. The terminal has a built-in microphone, camera, and speaker, and uses this hardware to collect audio and image data during consultations.

[0619] As a concrete example, if a user asks a home robot, "How can I relieve stress at work?", the robot records the utterance as audio on the device and sends it to a server. The server analyzes the emotions and generates specific advice to alleviate stress.

[0620] An example of a prompt used by a generative AI model is, "Think of the best thing to say to a user who is feeling stressed." Based on this prompt, the system can generate a thoughtful response.

[0621] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0622] Step 1:

[0623] User input

[0624] The user inputs their inquiry into the terminal via voice or text. The terminal acquires the voice or text data using a microphone and text input device. This input data forms the basis for the next processing step.

[0625] Step 2:

[0626] Sending data

[0627] The terminal sends the acquired voice and text data to the server. The transmission is performed using a secure protocol to maintain real-time data availability.

[0628] Step 3:

[0629] Speech and text analysis

[0630] The server analyzes the received data. First, the audio data is converted to text using speech recognition software. Next, this text data is analyzed by spaCy, a natural language processing engine, to extract important keywords and context. This analysis provides the information necessary for subsequent sentiment evaluation.

[0631] Step 4:

[0632] Assessment of emotional state

[0633] The server uses a TensorFlow-based emotion recognition module to evaluate the analyzed text data. This evaluation infers the user's emotional state from the text's vocabulary and speech tone. For example, emotions such as stress and sadness are identified. Based on the evaluation results, an optimal response tailored to the emotion is generated.

[0634] Step 5:

[0635] Generating the optimal response

[0636] Based on the evaluation of the emotional state, the server uses a generative AI model to generate a response using a prompt (e.g., "Please think of the best thing to say when you are feeling stressed."). This response is configured to be sensitive to the user's emotions.

[0637] Step 6:

[0638] Response forwarding and display

[0639] The generated response is sent to the terminal. The terminal presents this to the user as voice or text. The terminal's speaker and display are utilized to provide real-time responses, allowing the user to receive personalized support.

[0640] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0641] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0642] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0643] [Fourth Embodiment]

[0644] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0645] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0646] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0647] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0648] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0649] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0650] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0651] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0652] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0653] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0654] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0655] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0656] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0657] As a means of implementing this invention, a system for consulting with experts using an online platform is constructed. The system consists of three main components: a server, a user's terminal, and an expert's device.

[0658] Server operation

[0659] The server plays a central role in this system, managing the database, generating artificial intelligence agents, and coordinating with experts. First, the server receives registered user information and stores it in the database. Then, based on the user's input, it uses a natural language processing engine to analyze the consultation content and generate an appropriate artificial intelligence agent. This agent facilitates smooth conversations with the user while utilizing information relevant to the consultation content to select the most suitable expert from the expert database.

[0660] Terminal operation

[0661] The user's device is an internet-connected device that, after registration, allows the user to consult with an artificial intelligence agent. Users can input consultation details related to their needs and receive responses in real time. If expert advice is needed, the AI ​​agent collects the information, queries the server, and provides appropriate real-time feedback.

[0662] Specific example

[0663] For example, if a user wants to "consult about a mortgage," the AI ​​agent understands the user's inquiry and sends that information to the server. The server searches a database of experts and selects an appropriate expert in the financial field. Based on the expert's registered profile, the AI ​​agent provides the user with expert advice and solutions. The user can easily check the agent's response on their device.

[0664] This specific example demonstrates that users can efficiently seek expert advice, while experts can create an efficient consultation environment that allows them to respond to a larger number of inquiries.

[0665] The following describes the processing flow.

[0666] Step 1:

[0667] The server verifies the user's access on the online platform and displays the account registration screen. The user enters their personal information and the area of ​​consultation they wish to pursue, and submits the registration form.

[0668] Step 2:

[0669] The server receives registration information submitted by the user, verifies the input, and then saves it to the database. After saving is complete, an automatic confirmation email is sent to the user to notify them of the registration completion.

[0670] Step 3:

[0671] The server invokes a natural language processing engine based on registered user information to generate an artificial intelligence agent that responds to the user's request. This agent includes an optimal language model based on the user's requirements.

[0672] Step 4:

[0673] The user logs into the system using their device and initiates a consultation through an artificial intelligence agent. The consultation details entered by the user are immediately analyzed by the agent.

[0674] Step 5:

[0675] The server searches a database of experts based on the consultation details received via the artificial intelligence agent. An algorithm then selects the expert best suited to the user's needs.

[0676] Step 6:

[0677] The server sends information on selected experts to an artificial intelligence agent, which then retrieves advice from the experts in real time and notifies the user.

[0678] Step 7:

[0679] The user reviews the advice provided by the agent on their device and asks further questions as needed. The agent continues to respond appropriately to these questions.

[0680] Step 8:

[0681] After the consultation ends, the server provides the user with a feedback format and stores the collected feedback in a database. This can then be used to improve future artificial intelligence agents.

[0682] (Example 1)

[0683] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0684] Conventional consultation systems have made it difficult for users to quickly select the appropriate expert when they need professional advice. Furthermore, the lack of means to accurately understand the user's intentions and provide appropriate responses during the consultation process resulted in a significant amount of time and effort being required. To address these issues, a more efficient and user-friendly consultation system is needed.

[0685] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0686] In this invention, the server includes means for generating an artificial intelligence program based on the consultation content entered by the user via an online infrastructure, means for storing user information using a secure communication protocol, and means for analyzing the entered consultation content using a natural language processing device to identify relevant topics and keywords. This enables the server to quickly understand the consultation content from the user, immediately select an appropriate expert, and conduct consultations in real time.

[0687] "Online infrastructure" refers to the entire platform that users access and use to send and receive information via the internet.

[0688] "Users" refers to individual users who use the system to input their consultation details and receive advice and information from experts.

[0689] An "artificial intelligence program" refers to a program that analyzes the content of inquiries entered by users and provides appropriate responses or selects experts.

[0690] A "collection of expert information" refers to a database that compiles information on experts in various fields, and experts are selected based on this database.

[0691] A "secure communication protocol" refers to a means of communication that ensures the safe transmission and reception of data over the internet, and generally utilizes encryption technology.

[0692] A "natural language processing device" is a device equipped with technology to analyze the content of a user's inquiry into natural language and understand its content.

[0693] This invention constructs a consultation system that utilizes an online infrastructure to allow users to obtain expert advice. The system consists of three main components: a server, a user's terminal, and an expert's device.

[0694] The server plays a central role in receiving user inquiries and generating artificial intelligence programs. First, the server stores user information in a database using a secure communication protocol (e.g., HTTPS). When a user inputs their inquiry from their terminal, the server analyzes the content using a natural language processing system (e.g., a general-purpose natural language processing engine) to identify relevant topics and keywords. Based on this, the server utilizes a generative AI model to generate an artificial intelligence program tailored to the user's inquiry.

[0695] This artificial intelligence program is designed to provide instant answers to questions entered by users through their devices. Users can initiate consultations and receive answers via devices such as smartphones and personal computers. If expert advice is needed, the generated AI program selects a relevant expert from a database of expert information and retrieves that information via a server.

[0696] For example, if a user wants to "consult about home renovations," the server will select an expert in architecture or design and provide the user with specific advice. The user can immediately see the agent's response on their device and, if they have any questions, can use additional prompts to obtain further information.

[0697] As an example of a prompt, entering a message in the format of "I would like to consult with the AI ​​agent about a mortgage. Please select the appropriate expert and provide the user with the necessary information" will allow the user to receive a professional response tailored to their needs.

[0698] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0699] Step 1:

[0700] Users access the system using their device and enter their consultation details and personal information. The entered data is sent to the server. Specifically, users enter their name, contact information, and the content of their consultation in text format, and then press the send button on their device.

[0701] Step 2:

[0702] The server stores data received from the user in a database using a secure communication protocol. Here, the server encrypts the user data to prevent unauthorized external access. As output, user information is securely stored.

[0703] Step 3:

[0704] The server analyzes the stored consultation content using a natural language processing (NLP) system. The input is the text of the consultation content submitted by the user, and the server uses natural language processing techniques to extract the main topics and keywords. This analysis identifies the category and urgency of the consultation content. The output is a collection of the analyzed topics and keywords.

[0705] Step 4:

[0706] The server generates an artificial intelligence program based on the analysis results. Here, a response process tailored to the user's inquiry is designed using the generated AI model. The output is an AI-based response script tailored to the user's inquiry.

[0707] Step 5:

[0708] The server searches a database of expert information and selects the expert best suited to the analyzed consultation content. The input is the topic and keywords obtained in step 3, and the server runs an algorithm based on the expert profiles in the database. The output is the profile information of the selected expert.

[0709] Step 6:

[0710] The server feeds information from selected experts back into the artificial intelligence program, providing real-time advice to the user. Inputs include the profile information of the selected expert and the user's consultation topic. The server sends a response to the user's terminal, which the user can view on the screen and obtain further necessary information using prompts. Outputs are specific advice and suggestions displayed on the user's terminal.

[0711] (Application Example 1)

[0712] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0713] The problem this invention aims to solve is to enable users who require specialized knowledge regarding financial transactions and settlements to receive prompt and appropriate expert advice. Conventional systems have been cumbersome and inefficient in selecting experts. Furthermore, there is the problem of difficulty in finding the optimal solution for individual cases by utilizing expert advice.

[0714] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0715] In this invention, the server includes means for generating an artificial intelligence agent based on the content of a request sent by a user via an online platform, means for selecting the most suitable expert from an expert dataset using the artificial intelligence agent, and additional means for efficiently matching users with experts who specialize in financial transactions and settlements, and for providing expert advice from the experts. This enables users to effectively receive advice from experts and make optimal choices regarding transactions and settlements.

[0716] An "online platform" is a system that allows users to access various services and information via the internet.

[0717] An "artificial intelligence agent" is an automated computer program that analyzes user input, selects appropriate experts based on that input, and facilitates information exchange between users and experts.

[0718] A "specialist dataset" is a database containing numerous expert profiles, each of which records the expert's skills and experience.

[0719] "Automated response" refers to an immediate reply generated by an artificial intelligence agent in response to a user's request, and its content is intended to help the user solve their problem.

[0720] "Knowledge of financial transactions and settlements" refers to the specialized information and understanding required when selecting, contracting for, and using financial services and products.

[0721] "Expert advice" refers to practical and useful advice given by someone with expertise and experience in a particular field.

[0722] The system for implementing this invention consists of three basic components: a server, a user terminal, and a specialist device.

[0723] The server is the central element of the system, responsible for database management, artificial intelligence model generation, and collaboration with experts. The server first receives input from the user and analyzes it using a natural language processing engine. Based on the analysis results, an artificial intelligence agent is then generated. This agent interacts with the user, providing information to select the most suitable expert from the expert dataset. The selected expert then provides the user with expert advice on financial transactions and settlements.

[0724] The user's device is used to connect to the internet when the user seeks advice. The user inputs their inquiry into the system via the device and receives responses in real time. Furthermore, they can easily access information from experts based on a dataset of experts directly on their device. For example, if a user is struggling to choose a financial product, they can input their question via their smartphone and receive immediate advice from an expert.

[0725] The artificial intelligence agent within this system is built on a generative AI model and can facilitate smooth collaboration with experts through prompt messages. For example, the prompt message, "User is asking about investment advice on mutual funds. Identify the best expert to provide recommendations and explain the options," is input into the generative AI model, and an appropriate expert is selected based on it.

[0726] The servers and terminals utilize common cloud infrastructure and natural language processing libraries (e.g., NLTK, spaCy) to analyze user input and efficiently match them with experts, thereby facilitating optimal choices regarding transactions and payments.

[0727] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0728] Step 1:

[0729] The user's device receives the user's inquiry as input and sends it to the server. The input data is in text format and concerns inquiries about specific financial transactions or settlements. This data is transferred to the server as is.

[0730] Step 2:

[0731] The server uses a natural language processing engine to analyze the received text data. This analysis extracts keywords and intents from the consultation content. As a result of this analysis, the server outputs the specific consultation category and related topics.

[0732] Step 3:

[0733] The server generates an artificial intelligence agent using a generative AI model based on the analysis results. This agent searches an expert dataset using the generated prompt sentences. The prompt sentences include phrases like "User is asking about investment advice on mutual funds. Identify the best expert to provide recommendations and explain the options." which are generated based on the analysis results.

[0734] Step 4:

[0735] The server selects the most suitable expert from the generated prompt text and dataset. Using information from the expert dataset as input, the AI ​​agent outputs the most appropriate expert. The selected expert possesses the skills and experience best suited to the user's consultation.

[0736] Step 5:

[0737] The server retrieves advice from selected experts in real time and sends it to the user's terminal. The retrieved advice is output as structured text and notified to the user's terminal.

[0738] Step 6:

[0739] The user's device receives the consultation results and displays them to the user. The displayed content consists of suggestions and advice from experts, which the user can use to decide on their next course of action.

[0740] Step 7:

[0741] The user's device collects feedback from the user and sends it to the server. This feedback includes information about the user's satisfaction level and usefulness of the consultation, and is sent to the server as input.

[0742] Step 8:

[0743] The server processes user feedback to improve the artificial intelligence agent. The feedback data is analyzed and evaluated as input, helping to fine-tune and train the AI ​​model. This will enable more accurate matching and advice in future consultations.

[0744] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0745] This invention proposes a system that connects users with experts via an online platform, incorporating an emotion engine that recognizes the user's emotions. The system consists of a server, a user's terminal, and an expert's device, and is designed to ensure effective communication.

[0746] Server operation

[0747] The server receives inquiries from users and generates an artificial intelligence agent that understands the content. The agent uses a natural language processing engine to analyze the presented inquiries as text. Furthermore, an emotion engine extracts emotional nuances from the user's text and voice and evaluates the user's emotional state. Based on this data, the server selects the most appropriate response that takes emotions into consideration.

[0748] Terminal operation

[0749] On the user's device, users can input their inquiries via voice or text. The emotion engine analyzes the input voice tone and word choices to determine the user's emotional state. User input is directly transmitted to the server, and the agent's response is displayed in real time. This response is tailored to the user's emotional state, providing more accurate and empathetic support.

[0750] Collaboration with experts

[0751] If a user's question requires specialized knowledge, the AI ​​agent consults a database of experts and selects the appropriate expert. The selected expert can then use emotional information collected by the emotion engine to provide detailed advice tailored to the user's situation. This information supports the expert's judgment and enables more effective interaction.

[0752] Specific example

[0753] For example, if a user inputs "I want to talk about stress at work," the emotion engine detects the level of emotional tension from the input and voice. Based on this information, the system responds to the user in a gentle tone that is sensitive to their emotions. If necessary, a psychological counseling specialist is selected to provide more specific support. In this way, this system, which utilizes the emotion engine, reduces stress and provides a sense of security to users.

[0754] This configuration allows the consultation process between users and professionals to address emotional aspects, thus enabling more comprehensive support.

[0755] The following describes the processing flow.

[0756] Step 1:

[0757] The user accesses the online platform using their device and logs into their account. The user then enters the details of their inquiry via text or voice.

[0758] Step 2:

[0759] The server receives user input in real time and activates a natural language processing engine. It analyzes the input content and identifies the category of the inquiry.

[0760] Step 3:

[0761] The emotion engine analyzes the user's input text and voice, and evaluates the user's emotional state based on the words used and their tone of voice.

[0762] Step 4:

[0763] The server generates an appropriate artificial intelligence agent based on the results of natural language processing and sentiment analysis. The agent prepares the optimal response according to the user's inquiry and emotional state.

[0764] Step 5:

[0765] The server sends a response to the user's terminal via an agent. The user can view suggestions and advice displayed on the screen in real time.

[0766] Step 6:

[0767] If the consultation requires further specialized knowledge, the server will refer to a database of experts and select an expert that best suits the user's needs.

[0768] Step 7:

[0769] The selected experts can take into account the information obtained from sentiment analysis to provide more accurate and emotionally resonant advice. The server delivers the experts' responses to the user via an artificial intelligence agent.

[0770] Step 8:

[0771] We have created an environment where users can optionally provide feedback after completing a consultation. The server collects this feedback and uses it to improve future artificial intelligence agents.

[0772] (Example 2)

[0773] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0774] Generating responses that align with users' emotional needs in online consultation support systems is challenging due to the complex and diverse technologies involved. Conventional technologies are insufficient in accurately understanding users' emotional states and responding appropriately to their concerns. There is a need for a system that builds deep trust through emotionally empathetic responses, thereby achieving more effective support.

[0775] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0776] In this invention, the server includes means for receiving consultation content input from a user via a communication network, means for generating an artificial intelligence agent using a natural language processing engine to analyze the consultation content, and means for evaluating the user's emotional state using an emotion analysis engine. This enables automated responses and expert selection adapted to the user.

[0777] A "communication network" is a system that enables the transmission and reception of data between multiple devices.

[0778] "Consultation content" refers to information about problems or questions that users need to resolve.

[0779] A "natural language processing engine" is software that processes text data using technology that analyzes and understands human language.

[0780] An "artificial intelligence agent" is a program built to automatically perform specific tasks and has the ability to analyze user input and provide appropriate responses.

[0781] A "sentiment analysis engine" is a tool that extracts emotional nuances from text and audio to evaluate the user's emotional state.

[0782] "Automated response" refers to an answer to a user's question generated by an artificial intelligence agent.

[0783] A "specialist database" is a collection of data that compiles information on experts in various fields, and is used to select the appropriate expert.

[0784] An "algorithm" refers to a defined set of calculations or processing steps for solving a specific problem.

[0785] This invention is an online consultation system primarily consisting of a server, a user's terminal, and a specialist's device. The system aims to generate appropriate responses to consultations while considering the user's emotional state, and to match them with a specialist as needed.

[0786] The server processes user inquiries received via the communication network. These inquiries are input as text or audio and are first analyzed by a natural language processing engine. For example, spaCy, a publicly available natural language processing library, is used for this analysis. Based on the information obtained through the analysis, the server extracts the user's problem.

[0787] In parallel, the server uses an emotion analysis engine to extract emotional nuances from text and audio data. This analysis engine utilizes existing analysis tools capable of finely evaluating emotional tone. Based on the results of this emotion analysis, the server uses a generative AI model to generate an automated response in a tone that is empathetic to the user. This generation process utilizes the prompt message, "Generate a response that provides appropriate advice in a gentle tone, based on the user's inquiry and emotional state."

[0788] If a user's problem requires advanced expertise, the server consults a database of experts and selects the most relevant one. This database contains profile information based on each expert's skills and achievements.

[0789] On the user's device, user input is sent to the server in real time, and the response from the server is displayed immediately. This allows users to receive prompt support.

[0790] For example, if a user inputs "I want to talk about stress at work," the emotion analysis engine detects the level of emotional tension from the input and voice. Based on this information, the system provides the user with a gentle and empathetic response, and if necessary, selects a psychological counseling specialist to provide detailed support.

[0791] Thus, this system aims to provide more accurate and effective online consultations while taking into account the user's feelings.

[0792] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0793] Step 1:

[0794] The user enters their inquiry details using a terminal. Input can be via voice or text. In the case of voice input, the terminal acquires the voice data and converts it to text using speech recognition software. At this time, either voice or text is obtained as input data. For voice input, data is acquired through the microphone, and the text data is output via speech recognition.

[0795] Step 2:

[0796] The terminal sends the acquired text data to the server. HTTPS, a secure communication protocol, is used for this process. The input data is in text format and is sent to the server as is.

[0797] Step 3:

[0798] The server analyzes the received text data using a natural language processing engine. Specifically, it uses a natural language processing library to analyze grammatical structure and extract keywords. The input data is text, and the output provides extracted keywords and sentence structure information.

[0799] Step 4:

[0800] The server passes the analyzed data to the sentiment analysis engine, which evaluates the user's emotions. The sentiment analysis engine extracts emotional nuances from the text data and quantifies the emotional state. The input is the text data of the analysis results, and the output is an emotion score.

[0801] Step 5:

[0802] The server generates a response using a generative AI model based on the emotion score and text data. This process creates a response with the prompt set to "Generate a response that provides appropriate advice in a gentle tone, based on the user's inquiry and emotional state." The input is the emotion score and text data, and the output is the generated response text.

[0803] Step 6:

[0804] If the consultation requires specialized knowledge, the server will refer to a database of experts and select the appropriate expert. A database search algorithm is used to identify highly relevant expert profiles. The input is keywords related to the consultation, and the output is information on the selected expert.

[0805] Step 7:

[0806] The server sends the generated response and expert information to the terminal. The data is encrypted using the HTTPS protocol, allowing the user to receive results in real time.

[0807] Step 8:

[0808] The terminal displays the response received from the server to the user. The displayed content is output in a format adapted to the user's screen and can be checked immediately. This allows the user to receive quick and emotionally sensitive support.

[0809] (Application Example 2)

[0810] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0811] This invention aims to effectively recognize the emotional state of users and generate emotionally sensitive responses in a consultation system via an online platform. Conventional systems select experts based on user input, but struggle to provide responses that take emotional nuances into account, making it difficult to provide empathetic support to users.

[0812] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0813] In this invention, the server includes means for generating an artificial intelligence agent based on the consultation content entered by the user via an online platform, means for evaluating the user's emotional state with a module equipped with an emotion recognition function, and means for generating an emotion-sensitive response based on the evaluated emotional state. This enables empathetic and effective consultation that takes the user's emotions into consideration.

[0814] An "online platform" is a system that users can access via the internet and utilize various services and functions.

[0815] "User" refers to an individual or group that uses a service or system.

[0816] "Consultation content" refers to information about questions or problems that users present to experts or the system.

[0817] An "artificial intelligence agent" is a program that simulates intelligent tasks like those performed by humans, processing data and making decisions.

[0818] The "emotion recognition function" is a function that analyzes the user's language expression and tone of voice to determine their emotional state.

[0819] A "specialist database" is a data bank that compiles information on experts with knowledge and experience in a specific field.

[0820] "Real-time" refers to a system that responds quickly to user actions and processes them immediately.

[0821] "Generating a response" means preparing answers or reactions that the system will generate based on user input.

[0822] "Collecting feedback" means gathering opinions and evaluations from users and using them as material for improvement and evaluation.

[0823] The system for realizing this invention provides consultation services incorporating emotion recognition capabilities via an online platform, and is configured as follows.

[0824] The server runs an emotion recognition system built using TensorFlow. When a consultation request is entered, the server sends the voice and text data from the user to a cloud service. The transmitted data is analyzed using spaCy, a natural language processing engine, to evaluate the emotional state. Based on this evaluation, an optimal response that takes emotions into consideration is generated and provided to the user in real time.

[0825] The terminal receives user input and transfers the data to the server. The terminal has a built-in microphone, camera, and speaker, and uses this hardware to collect audio and image data during consultations.

[0826] As a concrete example, if a user asks a home robot, "How can I relieve stress at work?", the robot records the utterance as audio on the device and sends it to a server. The server analyzes the emotions and generates specific advice to alleviate stress.

[0827] An example of a prompt used by a generative AI model is, "Think of the best thing to say to a user who is feeling stressed." Based on this prompt, the system can generate a thoughtful response.

[0828] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0829] Step 1:

[0830] User input

[0831] The user inputs their inquiry into the terminal via voice or text. The terminal acquires the voice or text data using a microphone and text input device. This input data forms the basis for the next processing step.

[0832] Step 2:

[0833] Sending data

[0834] The terminal sends the acquired voice and text data to the server. The transmission is performed using a secure protocol to maintain real-time data availability.

[0835] Step 3:

[0836] Speech and text analysis

[0837] The server analyzes the received data. First, the audio data is converted to text using speech recognition software. Next, this text data is analyzed by spaCy, a natural language processing engine, to extract important keywords and context. This analysis provides the information necessary for subsequent sentiment evaluation.

[0838] Step 4:

[0839] Assessment of emotional state

[0840] The server uses a TensorFlow-based emotion recognition module to evaluate the analyzed text data. This evaluation infers the user's emotional state from the text's vocabulary and speech tone. For example, emotions such as stress and sadness are identified. Based on the evaluation results, an optimal response tailored to the emotion is generated.

[0841] Step 5:

[0842] Generating the optimal response

[0843] Based on the evaluation of the emotional state, the server uses a generative AI model to generate a response using a prompt (e.g., "Please think of the best thing to say when you are feeling stressed."). This response is configured to be sensitive to the user's emotions.

[0844] Step 6:

[0845] Response forwarding and display

[0846] The generated response is sent to the terminal. The terminal presents this to the user as voice or text. The terminal's speaker and display are utilized to provide real-time responses, allowing the user to receive personalized support.

[0847] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0848] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0849] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0850] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0851] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0852] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0853] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0854] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0855] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0856] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0857] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0858] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0859] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0860] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0861] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0862] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0863] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0864] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0865] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0866] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0867] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0868] The following is further disclosed regarding the embodiments described above.

[0869] (Claim 1)

[0870] A means for generating an artificial intelligence agent based on the consultation content entered by the user via an online platform,

[0871] A means for selecting an appropriate expert from an expert database using the aforementioned artificial intelligence agent,

[0872] A means of conducting real-time consultations between the user and a selected expert via the aforementioned artificial intelligence agent,

[0873] After the aforementioned consultation is completed, a means of collecting feedback from the user and using it to improve the performance of the artificial intelligence agent,

[0874] A system that includes this.

[0875] (Claim 2)

[0876] The system according to claim 1, wherein the artificial intelligence agent analyzes the user's input using a natural language processing engine and generates an automated response based on the content of the consultation.

[0877] (Claim 3)

[0878] The system according to claim 1, wherein the expert database includes a large number of expert profiles and comprises an algorithm for selecting experts based on the skills and experience associated with each profile.

[0879] "Example 1"

[0880] (Claim 1)

[0881] A means for generating an artificial intelligence program based on the consultation content entered by users via an online platform,

[0882] A means for selecting an appropriate expert from a set of expert information using the aforementioned artificial intelligence program,

[0883] A means of conducting real-time consultations between the user and a selected expert via the aforementioned artificial intelligence program,

[0884] After the aforementioned consultation is completed, a means is provided to collect responses from the user and use them to improve the performance of the artificial intelligence program.

[0885] Means for storing user information using secure communication protocols,

[0886] A means of analyzing the input consultation content using a natural language processing device and identifying related topics and keywords,

[0887] A means by which the generated artificial intelligence program provides a real-time response to the user,

[0888] A system that includes this.

[0889] (Claim 2)

[0890] The system according to claim 1, wherein the artificial intelligence program analyzes the user's input using a natural language processing device and generates an automated response based on the content of the consultation.

[0891] (Claim 3)

[0892] The system according to claim 1, wherein the set of expert information includes a large number of expert information items, and comprises an algorithm for selecting experts based on the technology and experience associated with each piece of information.

[0893] "Application Example 1"

[0894] (Claim 1)

[0895] A means for generating an artificial intelligence agent based on the content of a request sent by a user via an online platform,

[0896] A means for selecting the most suitable expert from an expert dataset using the aforementioned artificial intelligence agent,

[0897] A means of conducting real-time consultations between the user and a selected expert through the aforementioned artificial intelligence agent,

[0898] Following the aforementioned consultation, a means of collecting user feedback and utilizing it to improve the capabilities of the artificial intelligence agent was established.

[0899] An additional means to efficiently match users with experts specializing in financial transactions and settlements, and to provide expert advice from those experts,

[0900] A system that includes this.

[0901] (Claim 2)

[0902] The system according to claim 1, wherein the artificial intelligence agent interprets the user's transmitted content using a natural language processing engine and generates an automated response based on the content of the consultation.

[0903] (Claim 3)

[0904] The system according to claim 1, wherein the expert dataset has a large number of expert profiles and comprises a computational method for selecting experts based on the skills and experience associated with each profile.

[0905] "Example 2 of combining an emotion engine"

[0906] (Claim 1)

[0907] A means of receiving consultation content entered by users via a communication network,

[0908] A means for generating an artificial intelligence agent using a natural language processing engine to analyze the aforementioned consultation content,

[0909] A means of evaluating the emotional state of a user using an emotion analysis engine,

[0910] A means for generating an automated response adapted to the user based on the aforementioned analysis and emotion evaluation,

[0911] A means for selecting an appropriate expert from an expert database using the aforementioned automated response and sentiment data,

[0912] A means for conducting real-time consultations between a user and a selected expert via the aforementioned artificial intelligence agent,

[0913] A system that includes this.

[0914] (Claim 2)

[0915] The system according to claim 1, wherein the emotion analysis engine extracts emotional nuances from voice and text data and reflects them in the response.

[0916] (Claim 3)

[0917] The system according to claim 1, wherein the expert database includes a large amount of expert information and comprises an algorithm for selecting experts based on the technology and achievements related to each piece of information.

[0918] "Application example 2 when combining with an emotional engine"

[0919] (Claim 1)

[0920] A means for generating an artificial intelligence agent based on the consultation content entered by the user via an online platform,

[0921] A means for selecting an appropriate expert from an expert database using the aforementioned artificial intelligence agent,

[0922] A means of conducting real-time consultations between the user and a selected expert via the aforementioned artificial intelligence agent,

[0923] A means for evaluating a user's emotional state using a module equipped with emotion recognition capabilities,

[0924] A means for generating an emotionally sensitive response based on an evaluated emotional state,

[0925] After the aforementioned consultation is completed, a means of collecting feedback from the user and using it to improve the performance of the artificial intelligence agent,

[0926] A system that includes this.

[0927] (Claim 2)

[0928] The system according to claim 1, wherein the artificial intelligence agent analyzes the user's input using a natural language processing engine and generates an automated response based on the content of the consultation and the user's emotional state.

[0929] (Claim 3)

[0930] The system according to claim 1, wherein the expert database includes a large number of expert profiles and comprises an algorithm for selecting experts based on the abilities and experience associated with each profile. [Explanation of symbols]

[0931] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for generating an artificial intelligence agent based on the consultation content entered by the user via an online platform, A means for selecting an appropriate expert from an expert database using the aforementioned artificial intelligence agent, A means of conducting real-time consultations between the user and a selected expert via the aforementioned artificial intelligence agent, After the aforementioned consultation is completed, a means of collecting feedback from the user and using it to improve the performance of the artificial intelligence agent, A system that includes this.

2. The system according to claim 1, wherein the artificial intelligence agent analyzes the user's input using a natural language processing engine and generates an automated response based on the content of the consultation.

3. The system according to claim 1, wherein the expert database includes a large number of expert profiles and comprises an algorithm for selecting experts based on the skills and experience associated with each profile.

Images