system

Abstract

We provide the system. [Solution] A means of providing an interface for receiving input from the user, means for generating a search query based on the aforementioned input, A means for searching the database using the aforementioned query and obtaining the search results, A means for evaluating and matching the aforementioned search results and selecting the most suitable candidate, A means for presenting the user with information on the selected candidates and supporting the selection of a service that can be executed through a device with electronic access capabilities, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Currently, when a user searches for an agent specialized in a specific field or function, the information is scattered, so there is a problem that it is difficult to efficiently find the optimal agent. Therefore, there is a need for a system to quickly and accurately find the optimal agent according to the user's purpose from a huge number of agents.

Means for Solving the Problems

[0005] This invention provides means for receiving user input and generating search queries based on that input, thereby enabling the retrieval of agents matching user criteria from a database. Furthermore, it includes means for evaluating and matching search results and selecting the optimal candidate, enabling the user to effectively find agents that meet their objectives. Moreover, by using a database containing information on agents related to different specialized fields and employing an algorithm that takes past evaluation data and user criteria into consideration, it provides more accurate matching results.

[0006] "User input" refers to information about conditions and requests provided by the user through the system interface.

[0007] An "interface" refers to the screen or device that a user uses to interact with a system.

[0008] A "search query" is data that represents search conditions generated based on user input.

[0009] A "database" is a digital storage medium that systematically stores and allows retrieval of specific information.

[0010] "Evaluation and matching" is the process of selecting and ranking candidates based on specific criteria.

[0011] A "candidate" refers to a group of agents who may be selected based on specific criteria.

[0012] An "algorithm" is a method based on a series of computational steps and rules, used to solve a specific problem. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10] [[ID=2⑤]]It shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0014] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0019] In the following embodiments, a numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention provides a system that enables users to efficiently find agents suited to their specific purposes. First, the user enters criteria based on their needs through the terminal's interface. This interface is designed for ease of use, allowing users to easily specify search criteria. For example, if a user is looking for an agent with excellent speech recognition skills, they can enter this criterion and select relevant options. The terminal generates a search query from the entered information and sends this query to the server.

[0035] The server searches the database based on the received query and identifies agents that match the criteria. This database centrally manages agent information related to multiple specialties and is indexed to enable fast and accurate searches. Based on these search results, the server evaluates and matches agents. In the matching process, a proprietary algorithm based on past user evaluations and agent performance is used to select agents that are best suited to the user's requirements.

[0036] The server then sends information about potential agents back to the terminal, which visualizes this information for the user. The user can then view the presented list of candidates, compare the detailed information of each agent (e.g., function description, user ratings, cost, etc.), and make a final selection. This allows the user to efficiently find and use an agent that is perfectly suited to their specific purpose. Specifically, the system functions to support the user in selecting an agent from among multiple agents, for example, one that excels in speech recognition accuracy or cost-effectiveness.

[0037] The following describes the processing flow.

[0038] Step 1:

[0039] The user uses the terminal's interface to enter search criteria based on their needs (e.g., specific technologies or features).

[0040] Step 2:

[0041] The terminal checks the information received from the user, verifies its validity, and then generates a search query. This query includes the conditions specified by the user.

[0042] Step 3:

[0043] The terminal sends the generated search query to the server. This transmission is designed to be rapid, ensuring the server can receive the query properly.

[0044] Step 4:

[0045] Based on the query, the server searches the database containing agent information and creates a list of agents that match the criteria.

[0046] Step 5:

[0047] The server evaluates and matches agents based on the search results. This matching process uses an algorithm that takes into account past evaluations and agent characteristics.

[0048] Step 6:

[0049] The server selects the most suitable agent candidates and sends them to the terminal in a list format along with their detailed information.

[0050] Step 7:

[0051] The terminal displays a list of candidate agents to the user. Based on this information, the user can compare the characteristics and evaluations of each agent and make a selection.

[0052] (Example 1)

[0053] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0054] Current information retrieval systems make it difficult for users to efficiently find agents that match their specific purposes. In particular, there is a challenge in providing results that cover information from different specialized fields while also meeting the individual needs of the user quickly and accurately.

[0055] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0056] In this invention, the server includes means for generating search queries based on user input and transferring them from the terminal to the server; means for searching databases related to different specialized fields and extracting the search results; and means for using a generative AI model and utilizing a proprietary algorithm that takes into account the user's conditions and past evaluations in order to perform evaluation and matching. This makes it possible to efficiently acquire and present optimal agent information that perfectly matches the user's specific requirements.

[0057] A "terminal interface" is a display method designed with user-friendliness in mind to enable users to input information and interact with the system.

[0058] A "search query" is a conditional expression, based on user input, used to identify information within a database.

[0059] A "database" is a collection of information that aggregates and manages agent information across multiple specialized fields.

[0060] "Evaluation and matching" is the process of analyzing the acquired information based on the user's criteria and selecting the most suitable agent.

[0061] A "generative AI model" is an artificial intelligence technology that analyzes prompt text and past evaluation data to generate output that best matches the user's needs.

[0062] A "proprietary algorithm" is a specific calculation procedure that takes into account user conditions and past usage data to derive the optimal solution.

[0063] This system is a comprehensive information processing platform that helps users efficiently find agents that meet their specific needs. It primarily connects terminals and servers, providing them in a user-friendly format.

[0064] The user enters search criteria using the terminal's interface. This interface is intuitive and visually easy to understand, allowing users to input needs such as "an agent with high voice recognition accuracy" as prompt text. The terminal generates a search query based on the entered information and sends it to the server.

[0065] The server uses a high-performance database management system to search an indexed database. This database contains a wealth of information on agents across different areas of expertise. The server leverages a generative AI model and executes a proprietary algorithm to select the optimal agent that best suits the user's requirements. In doing so, it considers the user's past evaluation data and current conditions to provide personalized information.

[0066] For example, if the user enters "Please tell me the highest-rated speech recognition agent" as a prompt, the system will perform a search based on this and present the user with a list of the most suitable agents.

[0067] The device receives the search results and presents them to the user in a visualized format. Based on this information, the user can compare the characteristics of the agents and make the optimal choice. This allows the user to quickly find and use an agent that matches their criteria.

[0068] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0069] Step 1:

[0070] The user uses the terminal interface to enter prompt text as search criteria. Specifically, the user specifies concrete conditions such as "agents with high speech recognition accuracy." This input information is the starting data for processing. After receiving the input, the terminal prepares to generate the query.

[0071] Step 2:

[0072] The terminal parses the prompt received from the user and generates a search query. This query generation process uses natural language processing techniques to interpret the user's intent and convert it into a query format suitable for database searching. The output is an optimized search query ready to be passed to the server.

[0073] Step 3:

[0074] The server receives search queries sent from terminals. Based on these queries, it searches a high-performance database and extracts agent information that matches the input query. During the search process, the server utilizes indexing technology to enable rapid data retrieval. The output is a list of unevaluated candidate agents.

[0075] Step 4:

[0076] The server inputs the extracted agent list into a generating AI model for evaluation and matching. Here, a proprietary algorithm is applied that ranks agents based on past user ratings and agent performance data. The server then selects the most suitable agent candidates. The output is a list of agents ranked in descending order of evaluation.

[0077] Step 5:

[0078] The server sends optimized agent information to the terminal. The terminal receives this information and displays it in a format that is easily comparable to the user. The terminal uses visual elements to clearly indicate the characteristics and evaluation of each listed agent. The output is the final information presented to the user.

[0079] Step 6:

[0080] The user compares the agent information displayed on the terminal and makes a selection. Based on the detailed information provided, the user ultimately decides which agent best suits their needs. This is the user's final decision, utilizing the system's output.

[0081] (Application Example 1)

[0082] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0083] This solution addresses the challenge of quickly and accurately finding the optimal candidate that meets diverse criteria. In particular, the sheer volume of information and the complexity of searching make it difficult for users to make appropriate choices. This necessitates improving the situation where users are unable to efficiently select specific services or features.

[0084] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0085] In this invention, the server includes means for providing an interface to receive input from a user, means for generating search queries based on the input, means for searching a database using the queries and obtaining the search results, means for evaluating and matching the search results and selecting the optimal candidate, and means for presenting information on the selected candidate to the user and supporting the selection of a service that can be executed through a device with electronic access capabilities. This enables the user to efficiently identify the candidate best suited to their needs and utilize the information.

[0086] An "interface for receiving user input" is a mechanism for users to input conditions or requests through a terminal or device.

[0087] "Means of generating search queries" refers to the process of constructing instructions or questions for searching based on user input.

[0088] "Means for searching a database and obtaining the search results" refers to a function that searches for information within a database and collects results according to specified conditions.

[0089] "Methods for evaluation and matching to select the optimal candidate" refers to the process of ranking the obtained search results based on multiple criteria and determining the option deemed most appropriate.

[0090] "Means of presenting information on selected candidates to the user" refers to methods that display details of the most suitable candidate in a way that is easy for the user to understand, thereby assisting in the selection process.

[0091] A "device with electronic utilization capabilities" refers to a device that can operate and utilize digital information, such as a computer, smartphone, or tablet.

[0092] "Means of supporting service selection" refers to methods of supporting users in making the best decisions by providing information and functions.

[0093] To realize this invention, the system operates on devices with electronic capabilities, such as smartphones and tablets. Users input conditions through a provided interface, and this data is automatically generated as search queries. This process primarily utilizes a frontend based on React Native. Queries sent from the device are managed on the server using the Node.js and Flask frameworks.

[0094] The server searches the PostgreSQL database based on the received query and retrieves the appropriate information. This database stores information related to different specialized fields and is indexed to enable efficient searching. Search results are evaluated and matched using a proprietary algorithm based on historical evaluation data and user criteria.

[0095] After evaluation, the optimal candidate is selected, and the server sends that information back to the terminal. The terminal displays the acquired information in an easy-to-understand manner for the user, supporting the user in selecting a service. This process provides the best option that meets such conditions, for example, when a user is looking for an electronic payment service with low fees and a high point reward rate.

[0096] As a concrete example, a user might enter the following prompt into the application: "Based on the latest user ratings and point redemption rate data, tell me which electronic payment service offers the best deal." By entering something like this, the system gathers relevant information and presents the optimal option. This allows the user to efficiently find the service that best suits their needs.

[0097] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0098] Step 1:

[0099] Users input their conditions and requests through the device's interface. This input data is based on the user's selected settings and preferences and is performed using the touch interface of a smartphone or tablet. User input includes specific search instructions based on the selected conditions and preferences.

[0100] Step 2:

[0101] The terminal generates a search query based on user input. It parses the entered conditions and constructs a query accordingly. Here, the input content is processed as data within the program, and a query statement for searching is generated. In this step, a query in a specific data format is constructed based on user input using React Native.

[0102] Step 3:

[0103] The generated query is sent from the terminal to the server. The server receives this query and prepares to retrieve information from the database. The query is received on the server side using Node.js and Flask and processed for the next steps. The query serves as input used for a series of database search operations.

[0104] Step 4:

[0105] The server searches the PostgreSQL database using the received query. In this step, it uses the indexed information in the database to quickly identify results that match the query. The server searches for the data records that best fit the specified criteria and extracts the relevant results.

[0106] Step 5:

[0107] The server evaluates and matches the retrieved search results, taking into account past evaluation data and user-specified conditions. Each item included in the search results is evaluated according to set criteria and a proprietary algorithm. The server performs data calculations using statistical methods and weighting to determine the optimal choice.

[0108] Step 6:

[0109] The server selects the most suitable candidate based on the evaluation and matching results and sends information about it back to the terminal. The information about the selected candidate includes detailed descriptions and evaluation metrics. The server selects the information most valuable to the user and organizes it in a visually easy-to-understand format.

[0110] Step 7:

[0111] The terminal displays candidate information received from the server to the user. The user can view detailed candidate information on the screen and select a service. This information is listed on the interface, designed to allow the user to easily compare and consider options.

[0112] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0113] This invention relates to a system incorporating an emotion engine that recognizes the user's emotional state and reflects it in the search and matching process. When a user searches for a suitable agent using the terminal's interface, they can also provide the system with their current emotions. The terminal is equipped with sensors or a camera to acquire the user's voice and facial expression data in real time. While the user enters search criteria into the terminal, the emotion engine analyzes the tone of voice and facial expressions to identify the user's emotions. This emotion recognition result is then considered by the terminal when generating search queries.

[0114] The server receives a query that includes the user's emotional state and searches the database. The database contains diverse agent information, and the server performs new matching that takes emotional data into account. For example, if the user is feeling stressed, the emotion engine detects this and instructs the server to prioritize presenting agents that have functions to alleviate stress.

[0115] Subsequently, the server selects the most suitable agent candidates and sends them to the terminal. The terminal displays a list and presents the user with specific options. Including explanations and recommendations that reflect the user's emotions can improve user satisfaction.

[0116] For example, if a user feels anxious while planning a trip, the emotion engine will detect this and select an agent that offers an approach that provides relaxation and reassurance. On the other hand, if the user expresses positive emotions (e.g., excitement or anticipation), the system can present agents that offer more adventurous options or new suggestions. In this way, the system enables effective search and matching that leverages the user's emotional state.

[0117] The following describes the processing flow.

[0118] Step 1:

[0119] The user uses the device's interface to enter search criteria relevant to their purpose, and the system prepares to capture the user's emotional state. At this point, the device activates its camera and microphone and begins collecting the user's facial expressions and voice data.

[0120] Step 2:

[0121] The device transmits collected facial and voice data to an emotion engine, which analyzes the user's emotional state in real time. This analysis uses an emotion recognition algorithm to identify what emotions the user is experiencing.

[0122] Step 3:

[0123] The device obtains the emotion recognition results from the emotion engine and generates a search query based on them. The generated query reflects the user's input conditions and the recognized emotional state, and is ready to be sent to the server.

[0124] Step 4:

[0125] The terminal sends the generated search query to the server. Upon receiving the query, the server begins the process of listing potential agents from the database.

[0126] Step 5:

[0127] The server runs a matching algorithm that takes the user's emotional state into account and evaluates the most suitable agent candidates. For example, if stress is detected, it prioritizes agents that are suitable for stress relief.

[0128] Step 6:

[0129] The server compiles detailed information about the selected agent candidates and sends it to the terminal. This detailed information includes functions, evaluations, and reasons for recommendation.

[0130] Step 7:

[0131] The terminal displays a list of potential agents to the user, allowing them to compare the characteristics and advantages of each agent. Based on the displayed information, the user can select the most suitable agent.

[0132] (Example 2)

[0133] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0134] Conventional information retrieval systems provide uniform search results without considering the user's emotional state, which can lead to a lack of personalized suggestions tailored to the user's psychological condition and a decrease in user satisfaction. There is a need to solve this problem and provide more appropriate information that reflects the user's emotional state.

[0135] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0136] In this invention, the server includes means for acquiring the user's emotional state using voice data and facial expression data, means for generating search queries based on the emotional state, and means for performing evaluation and matching considering the emotional data and selecting the optimal information. This makes it possible to provide personalized search results and suggestions according to the user's emotional state.

[0137] A "display device" is a general term for hardware or software that provides an interface for receiving input from a user.

[0138] "Emotional state" refers to a psychological or emotional state identified based on the user's voice data and facial expression data.

[0139] "Audio data" refers to acoustic information obtained by recording or streaming user speech.

[0140] "Facial expression data" refers to image or video information used to record or analyze a user's facial expressions.

[0141] A "search query" refers to a string of characters or commands generated by a system to retrieve information, based on user input and emotional state.

[0142] "Storage device" refers to physical or virtual data storage for saving information.

[0143] "Evaluation and matching" refers to the process of analyzing the acquired search results based on sentiment data to select the information that is most suitable for the user.

[0144] The term "expert" refers to a source of information or an individual who possesses specialized knowledge and experience in a diverse field.

[0145] This invention relates to an information provision system that takes into account the user's psychological state. The terminal is equipped with sensors and cameras to acquire voice data and facial expression data from the user in real time. These hardware components consist of a high-performance microphone and camera that accurately capture the user's emotional state.

[0146] In addition to the search criteria entered by the user, the device activates an emotion engine to analyze the tone of voice and facial expressions. This emotion engine uses speech recognition and image processing software to identify and digitize the user's emotional state. The results of this analysis are used, along with the user's input, to form the search query.

[0147] The server receives queries sent from terminals and searches its database, which serves as its storage device. This database contains information on experts in various specialized fields. The server applies evaluation and matching algorithms that take sentiment data into consideration to select the most relevant information for the user.

[0148] For example, if a user is feeling anxious while planning a trip, the emotion engine will detect this and prioritize selecting experts who can provide relaxation and reassurance. This process allows users to receive personalized suggestions based on their emotional state.

[0149] An example of an input prompt to the generative AI model would be: "When the user is planning a trip, emotional data detects that they are feeling anxious. Therefore, please generate suggestions to reassure them." This allows for the provision of information that matches the user's needs.

[0150] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0151] Step 1:

[0152] The user enters their search objective through the device's interface.

[0153] The information entered includes text input and voice commands. This information is registered as input data. Specifically, users may enter a destination in the search bar or use voice recognition to give voice commands for vocabulary.

[0154] Step 2:

[0155] Sensors and cameras built into the device acquire the user's voice data and facial expression data in real time.

[0156] This data is used as input data to analyze the user's voice tone and facial expressions. The device uses a high-performance microphone to acquire voice and a camera to capture facial expressions. This data is then processed for analysis.

[0157] Step 3:

[0158] The device activates its emotion engine and analyzes the acquired voice and facial expression data.

[0159] The emotion engine applies a speech recognition system and image processing algorithms to identify the user's emotional state. Based on the input data, it analyzes emotions such as anxiety and joy from voice tone and also evaluates the emotional state from facial recognition. As a result, data indicating the emotional state is output.

[0160] Step 4:

[0161] The device generates a search query based on the user's search criteria and identified emotional state.

[0162] The generated queries also include user sentiment data, which is then sent to the next step. This query generation process utilizes a text generation algorithm to create prompts based on the sentiment data. The final output is a sentiment-aware search query.

[0163] Step 5:

[0164] The terminal sends the generated search query to the server.

[0165] The server receives this input and initiates a request to search the storage device. The terminal sends the query to the server over the network, and the data sent here becomes the input data for the search.

[0166] Step 6:

[0167] The server uses the received query to search the database, which is its storage device.

[0168] This search process utilizes a database search algorithm to extract relevant information while taking sentiment into account. By entering queries into the database, corresponding expert information is output.

[0169] Step 7:

[0170] The server selects the most relevant information, taking emotional data into consideration, and sends the output information to the terminal.

[0171] This involves using an agent selection algorithm to choose recommendations that reflect sentiment data. The selected information becomes the output data presented to the user.

[0172] Step 8:

[0173] The terminal displays information sent from the server to the user.

[0174] Information is displayed on the device in a list format or similar, and is provided in a way that allows the user to select from it. The outputted information is customized based on the user's emotional state.

[0175] (Application Example 2)

[0176] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0177] Traditional systems struggle to provide effective information that resonates with users' emotions because they match and present information without considering the user's emotional state. Furthermore, the inability to receive recommendations tailored to the user's current emotions limits the potential for improving the user experience. This can lead to decreased user satisfaction and an inability to provide flexible support that meets individual needs.

[0178] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0179] In this invention, the server includes means for providing a terminal device to receive input from a user, means for recognizing an emotional state from voice data and image data acquired using the terminal device, and means for generating search queries based on the input and emotional state. This makes it possible to analyze the user's emotions in real time and generate search queries accordingly, thereby presenting information that matches the user's emotions.

[0180] A "terminal device that receives input from a user" is an information processing device that is used by a user to input information and is equipped with an interface for acquiring audio data and image data.

[0181] "Means for recognizing emotional states from audio and image data" refers to algorithms and software that analyze acquired audio and image data to identify the user's emotions in real time.

[0182] "Means for generating search queries" refers to a process and apparatus for constructing queries for appropriate information retrieval based on user input data and recognized emotional states.

[0183] "Information storage" refers to a digital storage medium that records information about supporters related to different professional fields, making it accessible and searchable as needed.

[0184] "Evaluation and matching methods" refer to algorithms that process data, taking into account past evaluations, user conditions, and emotional states, in order to select the most suitable agent from among multiple candidates.

[0185] "Means for generating content including recommendations tailored to emotional state" refers to a device or program equipped with the function of creating and presenting information and suggestions optimized for a user based on the detected emotional state of that user.

[0186] This invention begins with the user providing input through a terminal device. The terminal is equipped with a camera and microphone, which are used to acquire audio and image data in real time. Emotion recognition uses an algorithm that analyzes the tone of voice and facial expressions. Specifically, it analyzes emotions by combining image analysis technology using OpenCV, for example, with an audio processing library. The results of this emotion analysis are sent to a server, where they are used together with text input from the user.

[0187] The server generates a search query that includes the received emotional state. Using this generated query, it searches the information storage and extracts appropriate candidates while considering relevant geographical information. The selection of candidates is determined by an algorithm that takes into account past evaluation data, user conditions, and detected emotional states.

[0188] The selected information is sent back to the device and presented to the user, including recommendations tailored to their emotional state. The generative AI model used in this process is capable of presenting the information the user currently desires in a highly personalized way.

[0189] For example, if a user shows a tired expression, the robot will be instructed to play relaxing music. Another example of a prompt using a generative AI model is, "When the user is smiling, please suggest some places to go out. Please update the suggestions in real time, including information about the surrounding area." In this way, the user experience is improved by utilizing the user's emotional state and providing the most suitable options for each individual situation.

[0190] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0191] Step 1:

[0192] The terminal receives user input. At this stage, the information received from the user includes audio and image data. Using the camera and microphone, this data is captured in real time and used for subsequent processing.

[0193] Step 2:

[0194] The device recognizes the user's emotional state from the acquired audio and image data. Specifically, it uses an audio processing library and image analysis techniques using OpenCV to perform voice tone analysis and facial expression recognition. As a result of this processing, the emotional state is output.

[0195] Step 3:

[0196] The terminal generates a search query based on the user's input and recognized emotional state. Because the user's emotional state is reflected in the search query generation, information retrieval optimized for the user becomes possible. This query is then forwarded to the server.

[0197] Step 4:

[0198] The server searches its information storage using the received search query. The information storage contains relevant support information. In this process, the server extracts the most suitable candidates, taking into account emotional states, geographical information, and past evaluation data.

[0199] Step 5:

[0200] The server adds recommendations based on the user's emotional state to the selected candidate information. Using a generative AI model, it generates content in the most beneficial and personalized way for the user. Here, additional information is generated based on a prompt (e.g., "Please suggest places to go when the user is smiling.").

[0201] Step 6:

[0202] The server then sends the final generated information to the terminal and presents it to the user. This allows the user to receive information optimized for their individual needs and current emotional state. This process improves the user experience.

[0203] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0204] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0205] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0206] [Second Embodiment]

[0207] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0208] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0209] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0210] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0211] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0212] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0213] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0214] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0215] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0216] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0217] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0218] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0219] This invention provides a system that enables users to efficiently find agents suited to their specific purposes. First, the user enters criteria based on their needs through the terminal's interface. This interface is designed for ease of use, allowing users to easily specify search criteria. For example, if a user is looking for an agent with excellent speech recognition skills, they can enter this criterion and select relevant options. The terminal generates a search query from the entered information and sends this query to the server.

[0220] The server searches the database based on the received query and identifies agents that match the criteria. This database centrally manages agent information related to multiple specialties and is indexed to enable fast and accurate searches. Based on these search results, the server evaluates and matches agents. In the matching process, a proprietary algorithm based on past user evaluations and agent performance is used to select agents that are best suited to the user's requirements.

[0221] The server then sends information about potential agents back to the terminal, which visualizes this information for the user. The user can then view the presented list of candidates, compare the detailed information of each agent (e.g., function description, user ratings, cost, etc.), and make a final selection. This allows the user to efficiently find and use an agent that is perfectly suited to their specific purpose. Specifically, the system functions to support the user in selecting an agent from among multiple agents, for example, one that excels in speech recognition accuracy or cost-effectiveness.

[0222] The following describes the processing flow.

[0223] Step 1:

[0224] The user uses the terminal's interface to enter search criteria based on their needs (e.g., specific technologies or features).

[0225] Step 2:

[0226] The terminal checks the information received from the user, verifies its validity, and then generates a search query. This query includes the conditions specified by the user.

[0227] Step 3:

[0228] The terminal sends the generated search query to the server. This transmission is designed to be rapid, ensuring the server can receive the query properly.

[0229] Step 4:

[0230] Based on the query, the server searches the database containing agent information and creates a list of agents that match the criteria.

[0231] Step 5:

[0232] The server evaluates and matches agents based on the search results. This matching process uses an algorithm that takes into account past evaluations and agent characteristics.

[0233] Step 6:

[0234] The server selects the most suitable agent candidates and sends them to the terminal in a list format along with their detailed information.

[0235] Step 7:

[0236] The terminal displays a list of candidate agents to the user. Based on this information, the user can compare the characteristics and evaluations of each agent and make a selection.

[0237] (Example 1)

[0238] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0239] Current information retrieval systems make it difficult for users to efficiently find agents that match their specific purposes. In particular, there is a challenge in providing results that cover information from different specialized fields while also meeting the individual needs of the user quickly and accurately.

[0240] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0241] In this invention, the server includes means for generating search queries based on user input and transferring them from the terminal to the server; means for searching databases related to different specialized fields and extracting the search results; and means for using a generative AI model and utilizing a proprietary algorithm that takes into account the user's conditions and past evaluations in order to perform evaluation and matching. This makes it possible to efficiently acquire and present optimal agent information that perfectly matches the user's specific requirements.

[0242] A "terminal interface" is a display method designed with user-friendliness in mind to enable users to input information and interact with the system.

[0243] A "search query" is a conditional expression, based on user input, used to identify information within a database.

[0244] A "database" is a collection of information that aggregates and manages agent information across multiple specialized fields.

[0245] "Evaluation and matching" is the process of analyzing the acquired information based on the user's criteria and selecting the most suitable agent.

[0246] A "generative AI model" is an artificial intelligence technology that analyzes prompt text and past evaluation data to generate output that best matches the user's needs.

[0247] A "proprietary algorithm" is a specific calculation procedure that takes into account user conditions and past usage data to derive the optimal solution.

[0248] This system is a comprehensive information processing platform that helps users efficiently find agents that meet their specific needs. It primarily connects terminals and servers, providing them in a user-friendly format.

[0249] The user enters search criteria using the terminal's interface. This interface is intuitive and visually easy to understand, allowing users to input needs such as "an agent with high voice recognition accuracy" as prompt text. The terminal generates a search query based on the entered information and sends it to the server.

[0250] The server uses a high-performance database management system to search an indexed database. This database contains a wealth of information on agents across different areas of expertise. The server leverages a generative AI model and executes a proprietary algorithm to select the optimal agent that best suits the user's requirements. In doing so, it considers the user's past evaluation data and current conditions to provide personalized information.

[0251] For example, if the user enters "Please tell me the highest-rated speech recognition agent" as a prompt, the system will perform a search based on this and present the user with a list of the most suitable agents.

[0252] The device receives the search results and presents them to the user in a visualized format. Based on this information, the user can compare the characteristics of the agents and make the optimal choice. This allows the user to quickly find and use an agent that matches their criteria.

[0253] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0254] Step 1:

[0255] The user uses the terminal interface to enter prompt text as search criteria. Specifically, the user specifies concrete conditions such as "agents with high speech recognition accuracy." This input information is the starting data for processing. After receiving the input, the terminal prepares to generate the query.

[0256] Step 2:

[0257] The terminal parses the prompt received from the user and generates a search query. This query generation process uses natural language processing techniques to interpret the user's intent and convert it into a query format suitable for database searching. The output is an optimized search query ready to be passed to the server.

[0258] Step 3:

[0259] The server receives search queries sent from terminals. Based on these queries, it searches a high-performance database and extracts agent information that matches the input query. During the search process, the server utilizes indexing technology to enable rapid data retrieval. The output is a list of unevaluated candidate agents.

[0260] Step 4:

[0261] The server inputs the extracted agent list into a generating AI model for evaluation and matching. Here, a proprietary algorithm is applied that ranks agents based on past user ratings and agent performance data. The server then selects the most suitable agent candidates. The output is a list of agents ranked in descending order of evaluation.

[0262] Step 5:

[0263] The server sends optimized agent information to the terminal. The terminal receives this information and displays it in a format that is easily comparable to the user. The terminal uses visual elements to clearly indicate the characteristics and evaluation of each listed agent. The output is the final information presented to the user.

[0264] Step 6:

[0265] The user compares the agent information displayed on the terminal and makes a selection. Based on the detailed information provided, the user ultimately decides which agent best suits their needs. This is the user's final decision, utilizing the system's output.

[0266] (Application Example 1)

[0267] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0268] This solution addresses the challenge of quickly and accurately finding the optimal candidate that meets diverse criteria. In particular, the sheer volume of information and the complexity of searching make it difficult for users to make appropriate choices. This necessitates improving the situation where users are unable to efficiently select specific services or features.

[0269] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0270] In this invention, the server includes means for providing an interface to receive input from a user, means for generating search queries based on the input, means for searching a database using the queries and obtaining the search results, means for evaluating and matching the search results and selecting the optimal candidate, and means for presenting information on the selected candidate to the user and supporting the selection of a service that can be executed through a device with electronic access capabilities. This enables the user to efficiently identify the candidate best suited to their needs and utilize the information.

[0271] An "interface for receiving user input" is a mechanism for users to input conditions or requests through a terminal or device.

[0272] "Means of generating search queries" refers to the process of constructing instructions or questions for searching based on user input.

[0273] "Means for searching a database and obtaining the search results" refers to a function that searches for information within a database and collects results according to specified conditions.

[0274] "Methods for evaluation and matching to select the optimal candidate" refers to the process of ranking the obtained search results based on multiple criteria and determining the option deemed most appropriate.

[0275] "Means of presenting information on selected candidates to the user" refers to methods that display details of the most suitable candidate in a way that is easy for the user to understand, thereby assisting in the selection process.

[0276] A "device with electronic utilization capabilities" refers to a device that can operate and utilize digital information, such as a computer, smartphone, or tablet.

[0277] "Means of supporting service selection" refers to methods of supporting users in making the best decisions by providing information and functions.

[0278] To realize this invention, the system operates on devices with electronic capabilities, such as smartphones and tablets. Users input conditions through a provided interface, and this data is automatically generated as search queries. This process primarily utilizes a frontend based on React Native. Queries sent from the device are managed on the server using the Node.js and Flask frameworks.

[0279] The server searches the PostgreSQL database based on the received query and retrieves the appropriate information. This database stores information related to different specialized fields and is indexed to enable efficient searching. Search results are evaluated and matched using a proprietary algorithm based on historical evaluation data and user criteria.

[0280] After evaluation, the optimal candidate is selected, and the server sends that information back to the terminal. The terminal displays the acquired information in an easy-to-understand manner for the user, supporting the user in selecting a service. This process provides the best option that meets such conditions, for example, when a user is looking for an electronic payment service with low fees and a high point reward rate.

[0281] As a concrete example, a user might enter the following prompt into the application: "Based on the latest user ratings and point redemption rate data, tell me which electronic payment service offers the best deal." By entering something like this, the system gathers relevant information and presents the optimal option. This allows the user to efficiently find the service that best suits their needs.

[0282] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0283] Step 1:

[0284] The user inputs their conditions and requirements through the interface of the terminal. This input data is based on the settings and preferences selected by the user and is performed using the touch interface of a smartphone or tablet. The user's input includes specific search instructions based on the selection of conditions and preferences.

[0285] Step 2:

[0286] Based on the input content from the user, the terminal generates a search query. It analyzes the input conditions and constructs a corresponding query. Here, the input content is processed as data within the program to generate a query statement for searching. In this step, a query in a specific data format is constructed based on the input from the user using React Native.

[0287] Step 3:

[0288] The generated query is sent from the terminal to the server. The server receives this query and prepares to search the information on the database. The query is received on the server side using Node.js and Flask and is processed for the next step. The query functions as the input for a series of database search operations.

[0289] Step 4:

[0290] The server searches the PostgreSQL database using the received query. In this step, the indexed information in the database is utilized to quickly identify the results that match the query. The server searches for the data records most suitable for the specified conditions and extracts the relevant results.

[0291] Step 5:

[0292] The server evaluates and matches the retrieved search results, taking into account past evaluation data and user-specified conditions. Each item included in the search results is evaluated according to set criteria and a proprietary algorithm. The server performs data calculations using statistical methods and weighting to determine the optimal choice.

[0293] Step 6:

[0294] The server selects the most suitable candidate based on the evaluation and matching results and sends information about it back to the terminal. The information about the selected candidate includes detailed descriptions and evaluation metrics. The server selects the information most valuable to the user and organizes it in a visually easy-to-understand format.

[0295] Step 7:

[0296] The terminal displays candidate information received from the server to the user. The user can view detailed candidate information on the screen and select a service. This information is listed on the interface, designed to allow the user to easily compare and consider options.

[0297] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0298] This invention relates to a system incorporating an emotion engine that recognizes the user's emotional state and reflects it in the search and matching process. When a user searches for a suitable agent using the terminal's interface, they can also provide the system with their current emotions. The terminal is equipped with sensors or a camera to acquire the user's voice and facial expression data in real time. While the user enters search criteria into the terminal, the emotion engine analyzes the tone of voice and facial expressions to identify the user's emotions. This emotion recognition result is then considered by the terminal when generating search queries.

[0299] The server receives a query that includes the user's emotional state and searches the database. The database contains diverse agent information, and the server performs new matching that takes emotional data into account. For example, if the user is feeling stressed, the emotion engine detects this and instructs the server to prioritize presenting agents that have functions to alleviate stress.

[0300] Subsequently, the server selects the most suitable agent candidates and sends them to the terminal. The terminal displays a list and presents the user with specific options. Including explanations and recommendations that reflect the user's emotions can improve user satisfaction.

[0301] For example, if a user feels anxious while planning a trip, the emotion engine will detect this and select an agent that offers an approach that provides relaxation and reassurance. On the other hand, if the user expresses positive emotions (e.g., excitement or anticipation), the system can present agents that offer more adventurous options or new suggestions. In this way, the system enables effective search and matching that leverages the user's emotional state.

[0302] The following describes the processing flow.

[0303] Step 1:

[0304] The user uses the device's interface to enter search criteria relevant to their purpose, and the system prepares to capture the user's emotional state. At this point, the device activates its camera and microphone and begins collecting the user's facial expressions and voice data.

[0305] Step 2:

[0306] The terminal sends the collected facial expression and voice data to the emotion engine to analyze the user's emotional state in real time. An emotion recognition algorithm is used in this analysis to identify what emotions the user is experiencing.

[0307] Step 3:

[0308] The terminal obtains the emotion recognition result from the emotion engine and generates a search query based on it. The generated query reflects the user's input conditions and the recognized emotional state, and is ready to be sent to the server.

[0309] Step 4:

[0310] The terminal sends the generated search query to the server. When the server receives the query, it starts the process of listing the candidate agents from the database.

[0311] Step 5:

[0312] The server executes a matching algorithm considering the user's emotional state and evaluates the candidate agents for the optimal one. For example, when the emotion of stress is detected, agents suitable for stress relief are prioritized.

[0313] Step 6:

[0314] The server organizes the detailed information of the selected agent candidates and sends it to the terminal. The detailed information includes functions, evaluations, recommended reasons, etc.

[0315] Step 7:

[0316] The terminal displays a list of agent candidates to the user so that the features and advantages of each agent can be compared. The user can select the most suitable agent based on the displayed information.

[0317] (Example 2)

[0318] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0319] Conventional information retrieval systems provide uniform search results without considering the user's emotional state, which can lead to a lack of personalized suggestions tailored to the user's psychological condition and a decrease in user satisfaction. There is a need to solve this problem and provide more appropriate information that reflects the user's emotional state.

[0320] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0321] In this invention, the server includes means for acquiring the user's emotional state using voice data and facial expression data, means for generating search queries based on the emotional state, and means for performing evaluation and matching considering the emotional data and selecting the optimal information. This makes it possible to provide personalized search results and suggestions according to the user's emotional state.

[0322] A "display device" is a general term for hardware or software that provides an interface for receiving input from a user.

[0323] "Emotional state" refers to a psychological or emotional state identified based on the user's voice data and facial expression data.

[0324] "Audio data" refers to acoustic information obtained by recording or streaming user speech.

[0325] "Facial expression data" refers to image or video information used to record or analyze a user's facial expressions.

[0326] A "search query" refers to a string of characters or commands generated by a system to retrieve information, based on user input and emotional state.

[0327] "Storage device" refers to physical or virtual data storage for saving information.

[0328] "Evaluation and matching" refers to the process of analyzing the acquired search results based on sentiment data to select the information that is most suitable for the user.

[0329] The term "expert" refers to a source of information or an individual who possesses specialized knowledge and experience in a diverse field.

[0330] This invention relates to an information provision system that takes into account the user's psychological state. The terminal is equipped with sensors and cameras to acquire voice data and facial expression data from the user in real time. These hardware components consist of a high-performance microphone and camera that accurately capture the user's emotional state.

[0331] In addition to the search criteria entered by the user, the device activates an emotion engine to analyze the tone of voice and facial expressions. This emotion engine uses speech recognition and image processing software to identify and digitize the user's emotional state. The results of this analysis are used, along with the user's input, to form the search query.

[0332] The server receives queries sent from terminals and searches its database, which serves as its storage device. This database contains information on experts in various specialized fields. The server applies evaluation and matching algorithms that take sentiment data into consideration to select the most relevant information for the user.

[0333] For example, if a user is feeling anxious while planning a trip, the emotion engine will detect this and prioritize selecting experts who can provide relaxation and reassurance. This process allows users to receive personalized suggestions based on their emotional state.

[0334] An example of an input prompt to the generative AI model would be: "When the user is planning a trip, emotional data detects that they are feeling anxious. Therefore, please generate suggestions to reassure them." This allows for the provision of information that matches the user's needs.

[0335] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0336] Step 1:

[0337] The user enters their search objective through the device's interface.

[0338] The information entered includes text input and voice commands. This information is registered as input data. Specifically, users may enter a destination in the search bar or use voice recognition to give voice commands for vocabulary.

[0339] Step 2:

[0340] Sensors and cameras built into the device acquire the user's voice data and facial expression data in real time.

[0341] This data is used as input data to analyze the user's voice tone and facial expressions. The device uses a high-performance microphone to acquire voice and a camera to capture facial expressions. This data is then processed for analysis.

[0342] Step 3:

[0343] The device activates its emotion engine and analyzes the acquired voice and facial expression data.

[0344] The emotion engine applies a speech recognition system and image processing algorithms to identify the user's emotional state. Based on the input data, it analyzes emotions such as anxiety and joy from voice tone and also evaluates the emotional state from facial recognition. As a result, data indicating the emotional state is output.

[0345] Step 4:

[0346] The device generates a search query based on the user's search criteria and identified emotional state.

[0347] The generated queries also include user sentiment data, which is then sent to the next step. This query generation process utilizes a text generation algorithm to create prompts based on the sentiment data. The final output is a sentiment-aware search query.

[0348] Step 5:

[0349] The terminal sends the generated search query to the server.

[0350] The server receives this input and initiates a request to search the storage device. The terminal sends the query to the server over the network, and the data sent here becomes the input data for the search.

[0351] Step 6:

[0352] The server uses the received query to search the database, which is its storage device.

[0353] This search process utilizes a database search algorithm to extract relevant information while taking sentiment into account. By entering queries into the database, corresponding expert information is output.

[0354] Step 7:

[0355] The server selects the most relevant information, taking emotional data into consideration, and sends the output information to the terminal.

[0356] This involves using an agent selection algorithm to choose recommendations that reflect sentiment data. The selected information becomes the output data presented to the user.

[0357] Step 8:

[0358] The terminal displays information sent from the server to the user.

[0359] Information is displayed on the device in a list format or similar, and is provided in a way that allows the user to select from it. The outputted information is customized based on the user's emotional state.

[0360] (Application Example 2)

[0361] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0362] Traditional systems struggle to provide effective information that resonates with users' emotions because they match and present information without considering the user's emotional state. Furthermore, the inability to receive recommendations tailored to the user's current emotions limits the potential for improving the user experience. This can lead to decreased user satisfaction and an inability to provide flexible support that meets individual needs.

[0363] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0364] In this invention, the server includes means for providing a terminal device to receive input from a user, means for recognizing an emotional state from voice data and image data acquired using the terminal device, and means for generating search queries based on the input and emotional state. This makes it possible to analyze the user's emotions in real time and generate search queries accordingly, thereby presenting information that matches the user's emotions.

[0365] A "terminal device that receives input from a user" is an information processing device that is used by a user to input information and is equipped with an interface for acquiring audio data and image data.

[0366] "Means for recognizing emotional states from audio and image data" refers to algorithms and software that analyze acquired audio and image data to identify the user's emotions in real time.

[0367] "Means for generating search queries" refers to a process and apparatus for constructing queries for appropriate information retrieval based on user input data and recognized emotional states.

[0368] "Information storage" refers to a digital storage medium that records information about supporters related to different professional fields, making it accessible and searchable as needed.

[0369] "Evaluation and matching methods" refer to algorithms that process data, taking into account past evaluations, user conditions, and emotional states, in order to select the most suitable agent from among multiple candidates.

[0370] "Means for generating content including recommendations tailored to emotional state" refers to a device or program equipped with the function of creating and presenting information and suggestions optimized for a user based on the detected emotional state of that user.

[0371] This invention begins with the user providing input through a terminal device. The terminal is equipped with a camera and microphone, which are used to acquire audio and image data in real time. Emotion recognition uses an algorithm that analyzes the tone of voice and facial expressions. Specifically, it analyzes emotions by combining image analysis technology using OpenCV, for example, with an audio processing library. The results of this emotion analysis are sent to a server, where they are used together with text input from the user.

[0372] The server generates a search query that includes the received emotional state. Using this generated query, it searches the information storage and extracts appropriate candidates while considering relevant geographical information. The selection of candidates is determined by an algorithm that takes into account past evaluation data, user conditions, and detected emotional states.

[0373] The selected information is sent back to the device and presented to the user, including recommendations tailored to their emotional state. The generative AI model used in this process is capable of presenting the information the user currently desires in a highly personalized way.

[0374] For example, if a user shows a tired expression, the robot will be instructed to play relaxing music. Another example of a prompt using a generative AI model is, "When the user is smiling, please suggest some places to go out. Please update the suggestions in real time, including information about the surrounding area." In this way, the user experience is improved by utilizing the user's emotional state and providing the most suitable options for each individual situation.

[0375] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0376] Step 1:

[0377] The terminal receives user input. At this stage, the information received from the user includes audio and image data. Using the camera and microphone, this data is captured in real time and used for subsequent processing.

[0378] Step 2:

[0379] The device recognizes the user's emotional state from the acquired audio and image data. Specifically, it uses an audio processing library and image analysis techniques using OpenCV to perform voice tone analysis and facial expression recognition. As a result of this processing, the emotional state is output.

[0380] Step 3:

[0381] The terminal generates a search query based on the user's input and recognized emotional state. Because the user's emotional state is reflected in the search query generation, information retrieval optimized for the user becomes possible. This query is then forwarded to the server.

[0382] Step 4:

[0383] The server searches its information storage using the received search query. The information storage contains relevant support information. In this process, the server extracts the most suitable candidates, taking into account emotional states, geographical information, and past evaluation data.

[0384] Step 5:

[0385] The server adds recommendations based on the user's emotional state to the selected candidate information. Using a generative AI model, it generates content in the most beneficial and personalized way for the user. Here, additional information is generated based on a prompt (e.g., "Please suggest places to go when the user is smiling.").

[0386] Step 6:

[0387] The server then sends the final generated information to the terminal and presents it to the user. This allows the user to receive information optimized for their individual needs and current emotional state. This process improves the user experience.

[0388] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0389] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0390] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0391] [Third Embodiment]

[0392] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0393] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0394] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0395] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0396] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0397] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0398] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0399] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0400] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0401] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0402] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0403] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0404] This invention provides a system that enables users to efficiently find agents suited to their specific purposes. First, the user enters criteria based on their needs through the terminal's interface. This interface is designed for ease of use, allowing users to easily specify search criteria. For example, if a user is looking for an agent with excellent speech recognition skills, they can enter this criterion and select relevant options. The terminal generates a search query from the entered information and sends this query to the server.

[0405] The server searches the database based on the received query and identifies agents that match the criteria. This database centrally manages agent information related to multiple specialties and is indexed to enable fast and accurate searches. Based on these search results, the server evaluates and matches agents. In the matching process, a proprietary algorithm based on past user evaluations and agent performance is used to select agents that are best suited to the user's requirements.

[0406] The server then sends information about potential agents back to the terminal, which visualizes this information for the user. The user can then view the presented list of candidates, compare the detailed information of each agent (e.g., function description, user ratings, cost, etc.), and make a final selection. This allows the user to efficiently find and use an agent that is perfectly suited to their specific purpose. Specifically, the system functions to support the user in selecting an agent from among multiple agents, for example, one that excels in speech recognition accuracy or cost-effectiveness.

[0407] The following describes the processing flow.

[0408] Step 1:

[0409] The user uses the terminal's interface to enter search criteria based on their needs (e.g., specific technologies or features).

[0410] Step 2:

[0411] The terminal checks the information received from the user, verifies its validity, and then generates a search query. This query includes the conditions specified by the user.

[0412] Step 3:

[0413] The terminal sends the generated search query to the server. This transmission is designed to be rapid, ensuring the server can receive the query properly.

[0414] Step 4:

[0415] Based on the query, the server searches the database containing agent information and creates a list of agents that match the criteria.

[0416] Step 5:

[0417] The server evaluates and matches agents based on the search results. This matching process uses an algorithm that takes into account past evaluations and agent characteristics.

[0418] Step 6:

[0419] The server selects the most suitable agent candidates and sends them to the terminal in a list format along with their detailed information.

[0420] Step 7:

[0421] The terminal displays a list of candidate agents to the user. Based on this information, the user can compare the characteristics and evaluations of each agent and make a selection.

[0422] (Example 1)

[0423] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0424] Current information retrieval systems make it difficult for users to efficiently find agents that match their specific purposes. In particular, there is a challenge in providing results that cover information from different specialized fields while also meeting the individual needs of the user quickly and accurately.

[0425] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0426] In this invention, the server includes means for generating search queries based on user input and transferring them from the terminal to the server; means for searching databases related to different specialized fields and extracting the search results; and means for using a generative AI model and utilizing a proprietary algorithm that takes into account the user's conditions and past evaluations in order to perform evaluation and matching. This makes it possible to efficiently acquire and present optimal agent information that perfectly matches the user's specific requirements.

[0427] A "terminal interface" is a display method designed with user-friendliness in mind to enable users to input information and interact with the system.

[0428] A "search query" is a conditional expression, based on user input, used to identify information within a database.

[0429] A "database" is a collection of information that aggregates and manages agent information across multiple specialized fields.

[0430] "Evaluation and matching" is the process of analyzing the acquired information based on the user's criteria and selecting the most suitable agent.

[0431] A "generative AI model" is an artificial intelligence technology that analyzes prompt text and past evaluation data to generate output that best matches the user's needs.

[0432] A "proprietary algorithm" is a specific calculation procedure that takes into account user conditions and past usage data to derive the optimal solution.

[0433] This system is a comprehensive information processing platform that helps users efficiently find agents that meet their specific needs. It primarily connects terminals and servers, providing them in a user-friendly format.

[0434] The user enters search criteria using the terminal's interface. This interface is intuitive and visually easy to understand, allowing users to input needs such as "an agent with high voice recognition accuracy" as prompt text. The terminal generates a search query based on the entered information and sends it to the server.

[0435] The server uses a high-performance database management system to search an indexed database. This database contains a wealth of information on agents across different areas of expertise. The server leverages a generative AI model and executes a proprietary algorithm to select the optimal agent that best suits the user's requirements. In doing so, it considers the user's past evaluation data and current conditions to provide personalized information.

[0436] For example, if the user enters "Please tell me the highest-rated speech recognition agent" as a prompt, the system will perform a search based on this and present the user with a list of the most suitable agents.

[0437] The device receives the search results and presents them to the user in a visualized format. Based on this information, the user can compare the characteristics of the agents and make the optimal choice. This allows the user to quickly find and use an agent that matches their criteria.

[0438] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0439] Step 1:

[0440] The user uses the terminal interface to enter prompt text as search criteria. Specifically, the user specifies concrete conditions such as "agents with high speech recognition accuracy." This input information is the starting data for processing. After receiving the input, the terminal prepares to generate the query.

[0441] Step 2:

[0442] The terminal parses the prompt received from the user and generates a search query. This query generation process uses natural language processing techniques to interpret the user's intent and convert it into a query format suitable for database searching. The output is an optimized search query ready to be passed to the server.

[0443] Step 3:

[0444] The server receives search queries sent from terminals. Based on these queries, it searches a high-performance database and extracts agent information that matches the input query. During the search process, the server utilizes indexing technology to enable rapid data retrieval. The output is a list of unevaluated candidate agents.

[0445] Step 4:

[0446] The server inputs the extracted agent list into a generating AI model for evaluation and matching. Here, a proprietary algorithm is applied that ranks agents based on past user ratings and agent performance data. The server then selects the most suitable agent candidates. The output is a list of agents ranked in descending order of evaluation.

[0447] Step 5:

[0448] The server sends optimized agent information to the terminal. The terminal receives this information and displays it in a format that is easily comparable to the user. The terminal uses visual elements to clearly indicate the characteristics and evaluation of each listed agent. The output is the final information presented to the user.

[0449] Step 6:

[0450] The user compares the agent information displayed on the terminal and makes a selection. Based on the detailed information provided, the user ultimately decides which agent best suits their needs. This is the user's final decision, utilizing the system's output.

[0451] (Application Example 1)

[0452] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0453] This solution addresses the challenge of quickly and accurately finding the optimal candidate that meets diverse criteria. In particular, the sheer volume of information and the complexity of searching make it difficult for users to make appropriate choices. This necessitates improving the situation where users are unable to efficiently select specific services or features.

[0454] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0455] In this invention, the server includes means for providing an interface to receive input from a user, means for generating search queries based on the input, means for searching a database using the queries and obtaining the search results, means for evaluating and matching the search results and selecting the optimal candidate, and means for presenting information on the selected candidate to the user and supporting the selection of a service that can be executed through a device with electronic access capabilities. This enables the user to efficiently identify the candidate best suited to their needs and utilize the information.

[0456] An "interface for receiving user input" is a mechanism for users to input conditions or requests through a terminal or device.

[0457] "Means of generating search queries" refers to the process of constructing instructions or questions for searching based on user input.

[0458] "Means for searching a database and obtaining the search results" refers to a function that searches for information within a database and collects results according to specified conditions.

[0459] "Methods for evaluation and matching to select the optimal candidate" refers to the process of ranking the obtained search results based on multiple criteria and determining the option deemed most appropriate.

[0460] "Means of presenting information on selected candidates to the user" refers to methods that display details of the most suitable candidate in a way that is easy for the user to understand, thereby assisting in the selection process.

[0461] A "device with electronic utilization capabilities" refers to a device that can operate and utilize digital information, such as a computer, smartphone, or tablet.

[0462] "Means of supporting service selection" refers to methods of supporting users in making the best decisions by providing information and functions.

[0463] To realize this invention, the system operates on devices with electronic capabilities, such as smartphones and tablets. Users input conditions through a provided interface, and this data is automatically generated as search queries. This process primarily utilizes a frontend based on React Native. Queries sent from the device are managed on the server using the Node.js and Flask frameworks.

[0464] The server searches the PostgreSQL database based on the received query and retrieves the appropriate information. This database stores information related to different specialized fields and is indexed to enable efficient searching. Search results are evaluated and matched using a proprietary algorithm based on historical evaluation data and user criteria.

[0465] After evaluation, the optimal candidate is selected, and the server sends that information back to the terminal. The terminal displays the acquired information in an easy-to-understand manner for the user, supporting the user in selecting a service. This process provides the best option that meets such conditions, for example, when a user is looking for an electronic payment service with low fees and a high point reward rate.

[0466] As a concrete example, a user might enter the following prompt into the application: "Based on the latest user ratings and point redemption rate data, tell me which electronic payment service offers the best deal." By entering something like this, the system gathers relevant information and presents the optimal option. This allows the user to efficiently find the service that best suits their needs.

[0467] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0468] Step 1:

[0469] Users input their conditions and requests through the device's interface. This input data is based on the user's selected settings and preferences and is performed using the touch interface of a smartphone or tablet. User input includes specific search instructions based on the selected conditions and preferences.

[0470] Step 2:

[0471] The terminal generates a search query based on user input. It parses the entered conditions and constructs a query accordingly. Here, the input content is processed as data within the program, and a query statement for searching is generated. In this step, a query in a specific data format is constructed based on user input using React Native.

[0472] Step 3:

[0473] The generated query is sent from the terminal to the server. The server receives this query and prepares to retrieve information from the database. The query is received on the server side using Node.js and Flask and processed for the next steps. The query serves as input used for a series of database search operations.

[0474] Step 4:

[0475] The server searches the PostgreSQL database using the received query. In this step, it uses the indexed information in the database to quickly identify results that match the query. The server searches for the data records that best fit the specified criteria and extracts the relevant results.

[0476] Step 5:

[0477] The server evaluates and matches the retrieved search results, taking into account past evaluation data and user-specified conditions. Each item included in the search results is evaluated according to set criteria and a proprietary algorithm. The server performs data calculations using statistical methods and weighting to determine the optimal choice.

[0478] Step 6:

[0479] The server selects the most suitable candidate based on the evaluation and matching results and sends information about it back to the terminal. The information about the selected candidate includes detailed descriptions and evaluation metrics. The server selects the information most valuable to the user and organizes it in a visually easy-to-understand format.

[0480] Step 7:

[0481] The terminal displays candidate information received from the server to the user. The user can view detailed candidate information on the screen and select a service. This information is listed on the interface, designed to allow the user to easily compare and consider options.

[0482] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0483] This invention relates to a system incorporating an emotion engine that recognizes the user's emotional state and reflects it in the search and matching process. When a user searches for a suitable agent using the terminal's interface, they can also provide the system with their current emotions. The terminal is equipped with sensors or a camera to acquire the user's voice and facial expression data in real time. While the user enters search criteria into the terminal, the emotion engine analyzes the tone of voice and facial expressions to identify the user's emotions. This emotion recognition result is then considered by the terminal when generating search queries.

[0484] The server receives a query that includes the user's emotional state and searches the database. The database contains diverse agent information, and the server performs new matching that takes emotional data into account. For example, if the user is feeling stressed, the emotion engine detects this and instructs the server to prioritize presenting agents that have functions to alleviate stress.

[0485] Subsequently, the server selects the most suitable agent candidates and sends them to the terminal. The terminal displays a list and presents the user with specific options. Including explanations and recommendations that reflect the user's emotions can improve user satisfaction.

[0486] For example, if a user feels anxious while planning a trip, the emotion engine will detect this and select an agent that offers an approach that provides relaxation and reassurance. On the other hand, if the user expresses positive emotions (e.g., excitement or anticipation), the system can present agents that offer more adventurous options or new suggestions. In this way, the system enables effective search and matching that leverages the user's emotional state.

[0487] The following describes the processing flow.

[0488] Step 1:

[0489] The user uses the device's interface to enter search criteria relevant to their purpose, and the system prepares to capture the user's emotional state. At this point, the device activates its camera and microphone and begins collecting the user's facial expressions and voice data.

[0490] Step 2:

[0491] The device transmits collected facial and voice data to an emotion engine, which analyzes the user's emotional state in real time. This analysis uses an emotion recognition algorithm to identify what emotions the user is experiencing.

[0492] Step 3:

[0493] The device obtains the emotion recognition results from the emotion engine and generates a search query based on them. The generated query reflects the user's input conditions and the recognized emotional state, and is ready to be sent to the server.

[0494] Step 4:

[0495] The terminal sends the generated search query to the server. Upon receiving the query, the server begins the process of listing potential agents from the database.

[0496] Step 5:

[0497] The server runs a matching algorithm that takes the user's emotional state into account and evaluates the most suitable agent candidates. For example, if stress is detected, it prioritizes agents that are suitable for stress relief.

[0498] Step 6:

[0499] The server compiles detailed information about the selected agent candidates and sends it to the terminal. This detailed information includes functions, evaluations, and reasons for recommendation.

[0500] Step 7:

[0501] The terminal displays a list of potential agents to the user, allowing them to compare the characteristics and advantages of each agent. Based on the displayed information, the user can select the most suitable agent.

[0502] (Example 2)

[0503] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0504] Conventional information retrieval systems provide uniform search results without considering the user's emotional state, which can lead to a lack of personalized suggestions tailored to the user's psychological condition and a decrease in user satisfaction. There is a need to solve this problem and provide more appropriate information that reflects the user's emotional state.

[0505] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0506] In this invention, the server includes means for acquiring the user's emotional state using voice data and facial expression data, means for generating search queries based on the emotional state, and means for performing evaluation and matching considering the emotional data and selecting the optimal information. This makes it possible to provide personalized search results and suggestions according to the user's emotional state.

[0507] A "display device" is a general term for hardware or software that provides an interface for receiving input from a user.

[0508] "Emotional state" refers to a psychological or emotional state identified based on the user's voice data and facial expression data.

[0509] "Audio data" refers to acoustic information obtained by recording or streaming user speech.

[0510] "Facial expression data" refers to image or video information used to record or analyze a user's facial expressions.

[0511] A "search query" refers to a string of characters or commands generated by a system to retrieve information, based on user input and emotional state.

[0512] "Storage device" refers to physical or virtual data storage for saving information.

[0513] "Evaluation and matching" refers to the process of analyzing the acquired search results based on sentiment data to select the information that is most suitable for the user.

[0514] The term "expert" refers to a source of information or an individual who possesses specialized knowledge and experience in a diverse field.

[0515] This invention relates to an information provision system that takes into account the user's psychological state. The terminal is equipped with sensors and cameras to acquire voice data and facial expression data from the user in real time. These hardware components consist of a high-performance microphone and camera that accurately capture the user's emotional state.

[0516] In addition to the search criteria entered by the user, the device activates an emotion engine to analyze the tone of voice and facial expressions. This emotion engine uses speech recognition and image processing software to identify and digitize the user's emotional state. The results of this analysis are used, along with the user's input, to form the search query.

[0517] The server receives queries sent from terminals and searches its database, which serves as its storage device. This database contains information on experts in various specialized fields. The server applies evaluation and matching algorithms that take sentiment data into consideration to select the most relevant information for the user.

[0518] For example, if a user is feeling anxious while planning a trip, the emotion engine will detect this and prioritize selecting experts who can provide relaxation and reassurance. This process allows users to receive personalized suggestions based on their emotional state.

[0519] An example of an input prompt to the generative AI model would be: "When the user is planning a trip, emotional data detects that they are feeling anxious. Therefore, please generate suggestions to reassure them." This allows for the provision of information that matches the user's needs.

[0520] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0521] Step 1:

[0522] The user enters their search objective through the device's interface.

[0523] The information entered includes text input and voice commands. This information is registered as input data. Specifically, users may enter a destination in the search bar or use voice recognition to give voice commands for vocabulary.

[0524] Step 2:

[0525] Sensors and cameras built into the device acquire the user's voice data and facial expression data in real time.

[0526] This data is used as input data to analyze the user's voice tone and facial expressions. The device uses a high-performance microphone to acquire voice and a camera to capture facial expressions. This data is then processed for analysis.

[0527] Step 3:

[0528] The device activates its emotion engine and analyzes the acquired voice and facial expression data.

[0529] The emotion engine applies a speech recognition system and image processing algorithms to identify the user's emotional state. Based on the input data, it analyzes emotions such as anxiety and joy from voice tone and also evaluates the emotional state from facial recognition. As a result, data indicating the emotional state is output.

[0530] Step 4:

[0531] The device generates a search query based on the user's search criteria and identified emotional state.

[0532] The generated queries also include user sentiment data, which is then sent to the next step. This query generation process utilizes a text generation algorithm to create prompts based on the sentiment data. The final output is a sentiment-aware search query.

[0533] Step 5:

[0534] The terminal sends the generated search query to the server.

[0535] The server receives this input and initiates a request to search the storage device. The terminal sends the query to the server over the network, and the data sent here becomes the input data for the search.

[0536] Step 6:

[0537] The server uses the received query to search the database, which is its storage device.

[0538] This search process utilizes a database search algorithm to extract relevant information while taking sentiment into account. By entering queries into the database, corresponding expert information is output.

[0539] Step 7:

[0540] The server selects the most relevant information, taking emotional data into consideration, and sends the output information to the terminal.

[0541] This involves using an agent selection algorithm to choose recommendations that reflect sentiment data. The selected information becomes the output data presented to the user.

[0542] Step 8:

[0543] The terminal displays information sent from the server to the user.

[0544] Information is displayed on the device in a list format or similar, and is provided in a way that allows the user to select from it. The outputted information is customized based on the user's emotional state.

[0545] (Application Example 2)

[0546] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0547] Traditional systems struggle to provide effective information that resonates with users' emotions because they match and present information without considering the user's emotional state. Furthermore, the inability to receive recommendations tailored to the user's current emotions limits the potential for improving the user experience. This can lead to decreased user satisfaction and an inability to provide flexible support that meets individual needs.

[0548] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0549] In this invention, the server includes means for providing a terminal device to receive input from a user, means for recognizing an emotional state from voice data and image data acquired using the terminal device, and means for generating search queries based on the input and emotional state. This makes it possible to analyze the user's emotions in real time and generate search queries accordingly, thereby presenting information that matches the user's emotions.

[0550] A "terminal device that receives input from a user" is an information processing device that is used by a user to input information and is equipped with an interface for acquiring audio data and image data.

[0551] "Means for recognizing emotional states from audio and image data" refers to algorithms and software that analyze acquired audio and image data to identify the user's emotions in real time.

[0552] "Means for generating search queries" refers to a process and apparatus for constructing queries for appropriate information retrieval based on user input data and recognized emotional states.

[0553] "Information storage" refers to a digital storage medium that records information about supporters related to different professional fields, making it accessible and searchable as needed.

[0554] "Evaluation and matching methods" refer to algorithms that process data, taking into account past evaluations, user conditions, and emotional states, in order to select the most suitable agent from among multiple candidates.

[0555] "Means for generating content including recommendations tailored to emotional state" refers to a device or program equipped with the function of creating and presenting information and suggestions optimized for a user based on the detected emotional state of that user.

[0556] This invention begins with the user providing input through a terminal device. The terminal is equipped with a camera and microphone, which are used to acquire audio and image data in real time. Emotion recognition uses an algorithm that analyzes the tone of voice and facial expressions. Specifically, it analyzes emotions by combining image analysis technology using OpenCV, for example, with an audio processing library. The results of this emotion analysis are sent to a server, where they are used together with text input from the user.

[0557] The server generates a search query that includes the received emotional state. Using this generated query, it searches the information storage and extracts appropriate candidates while considering relevant geographical information. The selection of candidates is determined by an algorithm that takes into account past evaluation data, user conditions, and detected emotional states.

[0558] The selected information is sent back to the device and presented to the user, including recommendations tailored to their emotional state. The generative AI model used in this process is capable of presenting the information the user currently desires in a highly personalized way.

[0559] For example, if a user shows a tired expression, the robot will be instructed to play relaxing music. Another example of a prompt using a generative AI model is, "When the user is smiling, please suggest some places to go out. Please update the suggestions in real time, including information about the surrounding area." In this way, the user experience is improved by utilizing the user's emotional state and providing the most suitable options for each individual situation.

[0560] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0561] Step 1:

[0562] The terminal receives user input. At this stage, the information received from the user includes audio and image data. Using the camera and microphone, this data is captured in real time and used for subsequent processing.

[0563] Step 2:

[0564] The device recognizes the user's emotional state from the acquired audio and image data. Specifically, it uses an audio processing library and image analysis techniques using OpenCV to perform voice tone analysis and facial expression recognition. As a result of this processing, the emotional state is output.

[0565] Step 3:

[0566] The terminal generates a search query based on the user's input and recognized emotional state. Because the user's emotional state is reflected in the search query generation, information retrieval optimized for the user becomes possible. This query is then forwarded to the server.

[0567] Step 4:

[0568] The server searches its information storage using the received search query. The information storage contains relevant support information. In this process, the server extracts the most suitable candidates, taking into account emotional states, geographical information, and past evaluation data.

[0569] Step 5:

[0570] The server adds recommendations based on the user's emotional state to the selected candidate information. Using a generative AI model, it generates content in the most beneficial and personalized way for the user. Here, additional information is generated based on a prompt (e.g., "Please suggest places to go when the user is smiling.").

[0571] Step 6:

[0572] The server then sends the final generated information to the terminal and presents it to the user. This allows the user to receive information optimized for their individual needs and current emotional state. This process improves the user experience.

[0573] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0574] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0575] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0576] [Fourth Embodiment]

[0577] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0578] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0579] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0580] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0581] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0582] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0583] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0584] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0585] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0586] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0587] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0588] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0589] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0590] This invention provides a system that enables users to efficiently find agents suited to their specific purposes. First, the user enters criteria based on their needs through the terminal's interface. This interface is designed for ease of use, allowing users to easily specify search criteria. For example, if a user is looking for an agent with excellent speech recognition skills, they can enter this criterion and select relevant options. The terminal generates a search query from the entered information and sends this query to the server.

[0591] The server searches the database based on the received query and identifies agents that match the criteria. This database centrally manages agent information related to multiple specialties and is indexed to enable fast and accurate searches. Based on these search results, the server evaluates and matches agents. In the matching process, a proprietary algorithm based on past user evaluations and agent performance is used to select agents that are best suited to the user's requirements.

[0592] The server then sends information about potential agents back to the terminal, which visualizes this information for the user. The user can then view the presented list of candidates, compare the detailed information of each agent (e.g., function description, user ratings, cost, etc.), and make a final selection. This allows the user to efficiently find and use an agent that is perfectly suited to their specific purpose. Specifically, the system functions to support the user in selecting an agent from among multiple agents, for example, one that excels in speech recognition accuracy or cost-effectiveness.

[0593] The following describes the processing flow.

[0594] Step 1:

[0595] The user uses the terminal's interface to enter search criteria based on their needs (e.g., specific technologies or features).

[0596] Step 2:

[0597] The terminal checks the information received from the user, verifies its validity, and then generates a search query. This query includes the conditions specified by the user.

[0598] Step 3:

[0599] The terminal sends the generated search query to the server. This transmission is designed to be rapid, ensuring the server can receive the query properly.

[0600] Step 4:

[0601] Based on the query, the server searches the database containing agent information and creates a list of agents that match the criteria.

[0602] Step 5:

[0603] The server evaluates and matches agents based on the search results. This matching process uses an algorithm that takes into account past evaluations and agent characteristics.

[0604] Step 6:

[0605] The server selects the most suitable agent candidates and sends them to the terminal in a list format along with their detailed information.

[0606] Step 7:

[0607] The terminal displays a list of candidate agents to the user. Based on this information, the user can compare the characteristics and evaluations of each agent and make a selection.

[0608] (Example 1)

[0609] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0610] Current information retrieval systems make it difficult for users to efficiently find agents that match their specific purposes. In particular, there is a challenge in providing results that cover information from different specialized fields while also meeting the individual needs of the user quickly and accurately.

[0611] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0612] In this invention, the server includes means for generating search queries based on user input and transferring them from the terminal to the server; means for searching databases related to different specialized fields and extracting the search results; and means for using a generative AI model and utilizing a proprietary algorithm that takes into account the user's conditions and past evaluations in order to perform evaluation and matching. This makes it possible to efficiently acquire and present optimal agent information that perfectly matches the user's specific requirements.

[0613] A "terminal interface" is a display method designed with user-friendliness in mind to enable users to input information and interact with the system.

[0614] A "search query" is a conditional expression, based on user input, used to identify information within a database.

[0615] A "database" is a collection of information that aggregates and manages agent information across multiple specialized fields.

[0616] "Evaluation and matching" is the process of analyzing the acquired information based on the user's criteria and selecting the most suitable agent.

[0617] A "generative AI model" is an artificial intelligence technology that analyzes prompt text and past evaluation data to generate output that best matches the user's needs.

[0618] A "proprietary algorithm" is a specific calculation procedure that takes into account user conditions and past usage data to derive the optimal solution.

[0619] This system is a comprehensive information processing platform that helps users efficiently find agents that meet their specific needs. It primarily connects terminals and servers, providing them in a user-friendly format.

[0620] The user enters search criteria using the terminal's interface. This interface is intuitive and visually easy to understand, allowing users to input needs such as "an agent with high voice recognition accuracy" as prompt text. The terminal generates a search query based on the entered information and sends it to the server.

[0621] The server uses a high-performance database management system to search an indexed database. This database contains a wealth of information on agents across different areas of expertise. The server leverages a generative AI model and executes a proprietary algorithm to select the optimal agent that best suits the user's requirements. In doing so, it considers the user's past evaluation data and current conditions to provide personalized information.

[0622] For example, if the user enters "Please tell me the highest-rated speech recognition agent" as a prompt, the system will perform a search based on this and present the user with a list of the most suitable agents.

[0623] The device receives the search results and presents them to the user in a visualized format. Based on this information, the user can compare the characteristics of the agents and make the optimal choice. This allows the user to quickly find and use an agent that matches their criteria.

[0624] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0625] Step 1:

[0626] The user uses the terminal interface to enter prompt text as search criteria. Specifically, the user specifies concrete conditions such as "agents with high speech recognition accuracy." This input information is the starting data for processing. After receiving the input, the terminal prepares to generate the query.

[0627] Step 2:

[0628] The terminal parses the prompt received from the user and generates a search query. This query generation process uses natural language processing techniques to interpret the user's intent and convert it into a query format suitable for database searching. The output is an optimized search query ready to be passed to the server.

[0629] Step 3:

[0630] The server receives search queries sent from terminals. Based on these queries, it searches a high-performance database and extracts agent information that matches the input query. During the search process, the server utilizes indexing technology to enable rapid data retrieval. The output is a list of unevaluated candidate agents.

[0631] Step 4:

[0632] The server inputs the extracted agent list into a generating AI model for evaluation and matching. Here, a proprietary algorithm is applied that ranks agents based on past user ratings and agent performance data. The server then selects the most suitable agent candidates. The output is a list of agents ranked in descending order of evaluation.

[0633] Step 5:

[0634] The server sends optimized agent information to the terminal. The terminal receives this information and displays it in a format that is easily comparable to the user. The terminal uses visual elements to clearly indicate the characteristics and evaluation of each listed agent. The output is the final information presented to the user.

[0635] Step 6:

[0636] The user compares the agent information displayed on the terminal and makes a selection. Based on the detailed information provided, the user ultimately decides which agent best suits their needs. This is the user's final decision, utilizing the system's output.

[0637] (Application Example 1)

[0638] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0639] This solution addresses the challenge of quickly and accurately finding the optimal candidate that meets diverse criteria. In particular, the sheer volume of information and the complexity of searching make it difficult for users to make appropriate choices. This necessitates improving the situation where users are unable to efficiently select specific services or features.

[0640] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0641] In this invention, the server includes means for providing an interface to receive input from a user, means for generating search queries based on the input, means for searching a database using the queries and obtaining the search results, means for evaluating and matching the search results and selecting the optimal candidate, and means for presenting information on the selected candidate to the user and supporting the selection of a service that can be executed through a device with electronic access capabilities. This enables the user to efficiently identify the candidate best suited to their needs and utilize the information.

[0642] An "interface for receiving user input" is a mechanism for users to input conditions or requests through a terminal or device.

[0643] "Means of generating search queries" refers to the process of constructing instructions or questions for searching based on user input.

[0644] "Means for searching a database and obtaining the search results" refers to a function that searches for information within a database and collects results according to specified conditions.

[0645] "Methods for evaluation and matching to select the optimal candidate" refers to the process of ranking the obtained search results based on multiple criteria and determining the option deemed most appropriate.

[0646] "Means of presenting information on selected candidates to the user" refers to methods that display details of the most suitable candidate in a way that is easy for the user to understand, thereby assisting in the selection process.

[0647] A "device with electronic utilization capabilities" refers to a device that can operate and utilize digital information, such as a computer, smartphone, or tablet.

[0648] "Means of supporting service selection" refers to methods of supporting users in making the best decisions by providing information and functions.

[0649] To realize this invention, the system operates on devices with electronic capabilities, such as smartphones and tablets. Users input conditions through a provided interface, and this data is automatically generated as search queries. This process primarily utilizes a frontend based on React Native. Queries sent from the device are managed on the server using the Node.js and Flask frameworks.

[0650] The server searches the PostgreSQL database based on the received query and retrieves the appropriate information. This database stores information related to different specialized fields and is indexed to enable efficient searching. Search results are evaluated and matched using a proprietary algorithm based on historical evaluation data and user criteria.

[0651] After evaluation, the optimal candidate is selected, and the server sends that information back to the terminal. The terminal displays the acquired information in an easy-to-understand manner for the user, supporting the user in selecting a service. This process provides the best option that meets such conditions, for example, when a user is looking for an electronic payment service with low fees and a high point reward rate.

[0652] As a concrete example, a user might enter the following prompt into the application: "Based on the latest user ratings and point redemption rate data, tell me which electronic payment service offers the best deal." By entering something like this, the system gathers relevant information and presents the optimal option. This allows the user to efficiently find the service that best suits their needs.

[0653] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0654] Step 1:

[0655] Users input their conditions and requests through the device's interface. This input data is based on the user's selected settings and preferences and is performed using the touch interface of a smartphone or tablet. User input includes specific search instructions based on the selected conditions and preferences.

[0656] Step 2:

[0657] The terminal generates a search query based on user input. It parses the entered conditions and constructs a query accordingly. Here, the input content is processed as data within the program, and a query statement for searching is generated. In this step, a query in a specific data format is constructed based on user input using React Native.

[0658] Step 3:

[0659] The generated query is sent from the terminal to the server. The server receives this query and prepares to retrieve information from the database. The query is received on the server side using Node.js and Flask and processed for the next steps. The query serves as input used for a series of database search operations.

[0660] Step 4:

[0661] The server searches the PostgreSQL database using the received query. In this step, it uses the indexed information in the database to quickly identify results that match the query. The server searches for the data records that best fit the specified criteria and extracts the relevant results.

[0662] Step 5:

[0663] The server evaluates and matches the retrieved search results, taking into account past evaluation data and user-specified conditions. Each item included in the search results is evaluated according to set criteria and a proprietary algorithm. The server performs data calculations using statistical methods and weighting to determine the optimal choice.

[0664] Step 6:

[0665] The server selects the most suitable candidate based on the evaluation and matching results and sends information about it back to the terminal. The information about the selected candidate includes detailed descriptions and evaluation metrics. The server selects the information most valuable to the user and organizes it in a visually easy-to-understand format.

[0666] Step 7:

[0667] The terminal displays candidate information received from the server to the user. The user can view detailed candidate information on the screen and select a service. This information is listed on the interface, designed to allow the user to easily compare and consider options.

[0668] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0669] This invention relates to a system incorporating an emotion engine that recognizes the user's emotional state and reflects it in the search and matching process. When a user searches for a suitable agent using the terminal's interface, they can also provide the system with their current emotions. The terminal is equipped with sensors or a camera to acquire the user's voice and facial expression data in real time. While the user enters search criteria into the terminal, the emotion engine analyzes the tone of voice and facial expressions to identify the user's emotions. This emotion recognition result is then considered by the terminal when generating search queries.

[0670] The server receives a query that includes the user's emotional state and searches the database. The database contains diverse agent information, and the server performs new matching that takes emotional data into account. For example, if the user is feeling stressed, the emotion engine detects this and instructs the server to prioritize presenting agents that have functions to alleviate stress.

[0671] Subsequently, the server selects the most suitable agent candidates and sends them to the terminal. The terminal displays a list and presents the user with specific options. Including explanations and recommendations that reflect the user's emotions can improve user satisfaction.

[0672] For example, if a user feels anxious while planning a trip, the emotion engine will detect this and select an agent that offers an approach that provides relaxation and reassurance. On the other hand, if the user expresses positive emotions (e.g., excitement or anticipation), the system can present agents that offer more adventurous options or new suggestions. In this way, the system enables effective search and matching that leverages the user's emotional state.

[0673] The following describes the processing flow.

[0674] Step 1:

[0675] The user uses the device's interface to enter search criteria relevant to their purpose, and the system prepares to capture the user's emotional state. At this point, the device activates its camera and microphone and begins collecting the user's facial expressions and voice data.

[0676] Step 2:

[0677] The device transmits collected facial and voice data to an emotion engine, which analyzes the user's emotional state in real time. This analysis uses an emotion recognition algorithm to identify what emotions the user is experiencing.

[0678] Step 3:

[0679] The device obtains the emotion recognition results from the emotion engine and generates a search query based on them. The generated query reflects the user's input conditions and the recognized emotional state, and is ready to be sent to the server.

[0680] Step 4:

[0681] The terminal sends the generated search query to the server. Upon receiving the query, the server begins the process of listing potential agents from the database.

[0682] Step 5:

[0683] The server runs a matching algorithm that takes the user's emotional state into account and evaluates the most suitable agent candidates. For example, if stress is detected, it prioritizes agents that are suitable for stress relief.

[0684] Step 6:

[0685] The server compiles detailed information about the selected agent candidates and sends it to the terminal. This detailed information includes functions, evaluations, and reasons for recommendation.

[0686] Step 7:

[0687] The terminal displays a list of potential agents to the user, allowing them to compare the characteristics and advantages of each agent. Based on the displayed information, the user can select the most suitable agent.

[0688] (Example 2)

[0689] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0690] Conventional information retrieval systems provide uniform search results without considering the user's emotional state, which can lead to a lack of personalized suggestions tailored to the user's psychological condition and a decrease in user satisfaction. There is a need to solve this problem and provide more appropriate information that reflects the user's emotional state.

[0691] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0692] In this invention, the server includes means for acquiring the user's emotional state using voice data and facial expression data, means for generating search queries based on the emotional state, and means for performing evaluation and matching considering the emotional data and selecting the optimal information. This makes it possible to provide personalized search results and suggestions according to the user's emotional state.

[0693] A "display device" is a general term for hardware or software that provides an interface for receiving input from a user.

[0694] "Emotional state" refers to a psychological or emotional state identified based on the user's voice data and facial expression data.

[0695] "Audio data" refers to acoustic information obtained by recording or streaming user speech.

[0696] "Facial expression data" refers to image or video information used to record or analyze a user's facial expressions.

[0697] A "search query" refers to a string of characters or commands generated by a system to retrieve information, based on user input and emotional state.

[0698] "Storage device" refers to physical or virtual data storage for saving information.

[0699] "Evaluation and matching" refers to the process of analyzing the acquired search results based on sentiment data to select the information that is most suitable for the user.

[0700] The term "expert" refers to a source of information or an individual who possesses specialized knowledge and experience in a diverse field.

[0701] This invention relates to an information provision system that takes into account the user's psychological state. The terminal is equipped with sensors and cameras to acquire voice data and facial expression data from the user in real time. These hardware components consist of a high-performance microphone and camera that accurately capture the user's emotional state.

[0702] In addition to the search criteria entered by the user, the device activates an emotion engine to analyze the tone of voice and facial expressions. This emotion engine uses speech recognition and image processing software to identify and digitize the user's emotional state. The results of this analysis are used, along with the user's input, to form the search query.

[0703] The server receives queries sent from terminals and searches its database, which serves as its storage device. This database contains information on experts in various specialized fields. The server applies evaluation and matching algorithms that take sentiment data into consideration to select the most relevant information for the user.

[0704] For example, if a user is feeling anxious while planning a trip, the emotion engine will detect this and prioritize selecting experts who can provide relaxation and reassurance. This process allows users to receive personalized suggestions based on their emotional state.

[0705] An example of an input prompt to the generative AI model would be: "When the user is planning a trip, emotional data detects that they are feeling anxious. Therefore, please generate suggestions to reassure them." This allows for the provision of information that matches the user's needs.

[0706] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0707] Step 1:

[0708] The user enters their search objective through the device's interface.

[0709] The information entered includes text input and voice commands. This information is registered as input data. Specifically, users may enter a destination in the search bar or use voice recognition to give voice commands for vocabulary.

[0710] Step 2:

[0711] Sensors and cameras built into the device acquire the user's voice data and facial expression data in real time.

[0712] This data is used as input data to analyze the user's voice tone and facial expressions. The device uses a high-performance microphone to acquire voice and a camera to capture facial expressions. This data is then processed for analysis.

[0713] Step 3:

[0714] The device activates its emotion engine and analyzes the acquired voice and facial expression data.

[0715] The emotion engine applies a speech recognition system and image processing algorithms to identify the user's emotional state. Based on the input data, it analyzes emotions such as anxiety and joy from voice tone and also evaluates the emotional state from facial recognition. As a result, data indicating the emotional state is output.

[0716] Step 4:

[0717] The device generates a search query based on the user's search criteria and identified emotional state.

[0718] The generated queries also include user sentiment data, which is then sent to the next step. This query generation process utilizes a text generation algorithm to create prompts based on the sentiment data. The final output is a sentiment-aware search query.

[0719] Step 5:

[0720] The terminal sends the generated search query to the server.

[0721] The server receives this input and initiates a request to search the storage device. The terminal sends the query to the server over the network, and the data sent here becomes the input data for the search.

[0722] Step 6:

[0723] The server uses the received query to search the database, which is its storage device.

[0724] This search process utilizes a database search algorithm to extract relevant information while taking sentiment into account. By entering queries into the database, corresponding expert information is output.

[0725] Step 7:

[0726] The server selects the most relevant information, taking emotional data into consideration, and sends the output information to the terminal.

[0727] This involves using an agent selection algorithm to choose recommendations that reflect sentiment data. The selected information becomes the output data presented to the user.

[0728] Step 8:

[0729] The terminal displays information sent from the server to the user.

[0730] Information is displayed on the device in a list format or similar, and is provided in a way that allows the user to select from it. The outputted information is customized based on the user's emotional state.

[0731] (Application Example 2)

[0732] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0733] Traditional systems struggle to provide effective information that resonates with users' emotions because they match and present information without considering the user's emotional state. Furthermore, the inability to receive recommendations tailored to the user's current emotions limits the potential for improving the user experience. This can lead to decreased user satisfaction and an inability to provide flexible support that meets individual needs.

[0734] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0735] In this invention, the server includes means for providing a terminal device to receive input from a user, means for recognizing an emotional state from voice data and image data acquired using the terminal device, and means for generating search queries based on the input and emotional state. This makes it possible to analyze the user's emotions in real time and generate search queries accordingly, thereby presenting information that matches the user's emotions.

[0736] A "terminal device that receives input from a user" is an information processing device that is used by a user to input information and is equipped with an interface for acquiring audio data and image data.

[0737] "Means for recognizing emotional states from audio and image data" refers to algorithms and software that analyze acquired audio and image data to identify the user's emotions in real time.

[0738] "Means for generating search queries" refers to a process and apparatus for constructing queries for appropriate information retrieval based on user input data and recognized emotional states.

[0739] "Information storage" refers to a digital storage medium that records information about supporters related to different professional fields, making it accessible and searchable as needed.

[0740] "Evaluation and matching methods" refer to algorithms that process data, taking into account past evaluations, user conditions, and emotional states, in order to select the most suitable agent from among multiple candidates.

[0741] "Means for generating content including recommendations tailored to emotional state" refers to a device or program equipped with the function of creating and presenting information and suggestions optimized for a user based on the detected emotional state of that user.

[0742] This invention begins with the user providing input through a terminal device. The terminal is equipped with a camera and microphone, which are used to acquire audio and image data in real time. Emotion recognition uses an algorithm that analyzes the tone of voice and facial expressions. Specifically, it analyzes emotions by combining image analysis technology using OpenCV, for example, with an audio processing library. The results of this emotion analysis are sent to a server, where they are used together with text input from the user.

[0743] The server generates a search query that includes the received emotional state. Using this generated query, it searches the information storage and extracts appropriate candidates while considering relevant geographical information. The selection of candidates is determined by an algorithm that takes into account past evaluation data, user conditions, and detected emotional states.

[0744] The selected information is sent back to the device and presented to the user, including recommendations tailored to their emotional state. The generative AI model used in this process is capable of presenting the information the user currently desires in a highly personalized way.

[0745] For example, if a user shows a tired expression, the robot will be instructed to play relaxing music. Another example of a prompt using a generative AI model is, "When the user is smiling, please suggest some places to go out. Please update the suggestions in real time, including information about the surrounding area." In this way, the user experience is improved by utilizing the user's emotional state and providing the most suitable options for each individual situation.

[0746] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0747] Step 1:

[0748] The terminal receives user input. At this stage, the information received from the user includes audio and image data. Using the camera and microphone, this data is captured in real time and used for subsequent processing.

[0749] Step 2:

[0750] The device recognizes the user's emotional state from the acquired audio and image data. Specifically, it uses an audio processing library and image analysis techniques using OpenCV to perform voice tone analysis and facial expression recognition. As a result of this processing, the emotional state is output.

[0751] Step 3:

[0752] The terminal generates a search query based on the user's input and recognized emotional state. Because the user's emotional state is reflected in the search query generation, information retrieval optimized for the user becomes possible. This query is then forwarded to the server.

[0753] Step 4:

[0754] The server searches its information storage using the received search query. The information storage contains relevant support information. In this process, the server extracts the most suitable candidates, taking into account emotional states, geographical information, and past evaluation data.

[0755] Step 5:

[0756] The server adds recommendations based on the user's emotional state to the selected candidate information. Using a generative AI model, it generates content in the most beneficial and personalized way for the user. Here, additional information is generated based on a prompt (e.g., "Please suggest places to go when the user is smiling.").

[0757] Step 6:

[0758] The server then sends the final generated information to the terminal and presents it to the user. This allows the user to receive information optimized for their individual needs and current emotional state. This process improves the user experience.

[0759] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0760] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0761] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0762] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0763] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. In the upper and lower directions of the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. Also, the upper side of the concentric circles is where "pleasant" emotions are located, and the lower side is where "unpleasant" emotions are located. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0764] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0765] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0766] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0767] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0768] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0769] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0770] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0771] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0772] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0773] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0774] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0775] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0776] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0777] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0778] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0779] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0780] The following is further disclosed regarding the embodiments described above.

[0781] (Claim 1)

[0782] A means of providing an interface for receiving input from the user,

[0783] means for generating a search query based on the aforementioned input,

[0784] A means for searching the database using the aforementioned query and obtaining the search results,

[0785] A means for evaluating and matching the aforementioned search results and selecting the most suitable candidate,

[0786] A means for presenting information on the selected candidates to the user,

[0787] A system that includes this.

[0788] (Claim 2)

[0789] The system according to claim 1, characterized in that the database stores information on agents related to different fields of expertise.

[0790] (Claim 3)

[0791] The system according to claim 1, characterized in that the evaluation and matching means uses an algorithm that takes into account the agent's past evaluation data and the user's conditions.

[0792] "Example 1"

[0793] (Claim 1)

[0794] A means of providing an interface for a terminal that receives input from a user,

[0795] A means for generating a search query based on the aforementioned input and transferring it from the terminal to the server,

[0796] A means for the server to search the database using the aforementioned query and extract the search results,

[0797] A means for performing evaluation and matching of the extracted search results on a server and ranking the most suitable candidates,

[0798] A means for presenting the ranked candidate information to the user via a terminal,

[0799] Using a proprietary algorithm, we utilize a generative AI model, taking into account past evaluation data and user conditions.

[0800] A system that includes this.

[0801] (Claim 2)

[0802] The system according to claim 1, characterized in that the database aggregates and holds information on a variety of agents related to different fields of expertise.

[0803] (Claim 3)

[0804] The system according to claim 1, characterized in that the evaluation and matching means analyzes prompt sentences using a generating AI model and employs an advanced algorithm that takes into account the agent's past evaluation data and conditions specified by the user.

[0805] "Application Example 1"

[0806] (Claim 1)

[0807] A means of providing an interface for receiving input from the user,

[0808] means for generating a search query based on the aforementioned input,

[0809] A means for searching the database using the aforementioned query and obtaining the search results,

[0810] A means for evaluating and matching the aforementioned search results and selecting the most suitable candidate,

[0811] A means for presenting the user with information on the selected candidates and supporting the selection of a service that can be executed through a device with electronic access capabilities,

[0812] A system that includes this.

[0813] (Claim 2)

[0814] The system according to claim 1, characterized in that the database stores information related to different specialized fields and supports selection based on a specific field.

[0815] (Claim 3)

[0816] The system according to claim 1, characterized in that the evaluation and matching means uses an algorithm that takes into account past evaluation data and user conditions, and evaluates the selected information according to criteria set by the applicant.

[0817] "Example 2 of combining an emotion engine"

[0818] (Claim 1)

[0819] Means for providing a display device that receives input from a user,

[0820] A means for acquiring the user's emotional state using voice data and facial expression data,

[0821] A means for generating a search query based on the aforementioned emotional state,

[0822] A means for searching a storage device using the generated query and obtaining the search results,

[0823] A means of selecting the optimal information by performing evaluation and matching that takes emotional data into consideration,

[0824] Means for providing the selected information to the user,

[0825] A system that includes this.

[0826] (Claim 2)

[0827] The system according to claim 1, characterized in that the storage device stores information on experts related to various specialized fields.

[0828] (Claim 3)

[0829] The system according to claim 1, characterized in that the evaluation and matching means uses an algorithm that takes into account the past evaluation data of experts and the emotional state of the user.

[0830] "Application example 2 when combining with an emotional engine"

[0831] (Claim 1)

[0832] Means for providing a terminal device that receives input from a user,

[0833] A means for recognizing emotional states from audio data and image data acquired using the aforementioned terminal device,

[0834] means for generating a search query based on the aforementioned input and emotional state,

[0835] A means for searching information storage using the aforementioned query and obtaining the search results,

[0836] A means for evaluating and matching the aforementioned search results and selecting the most suitable candidate,

[0837] A means for presenting information on the selected candidates to the user and generating content including recommendations tailored to their emotional state,

[0838] A system that includes this.

[0839] (Claim 2)

[0840] The system according to claim 1, characterized in that the information storage stores information on supporters related to different professional fields.

[0841] (Claim 3)

[0842] The system according to claim 1, characterized in that the evaluation and matching means uses an algorithm that takes into account the supporter's past evaluation data, the user's conditions, and their emotional state. [Explanation of Symbols]

[0843] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of providing an interface for receiving input from the user, means for generating a search query based on the aforementioned input, A means for searching the database using the aforementioned query and obtaining the search results, A means for evaluating and matching the aforementioned search results and selecting the most suitable candidate, A means for presenting the user with information on the selected candidates and supporting the selection of a service that can be executed through a device with electronic access capabilities, A system that includes this.

2. The system according to claim 1, characterized in that the database stores information related to different specialized fields and supports selection based on a specific field.

3. The system according to claim 1, characterized in that the evaluation and matching means uses an algorithm that takes into account past evaluation data and user conditions, and evaluates the selected information according to criteria set by the applicant.

Images