system

Abstract

We provide the system. [Solution] A means for setting up an information storage unit for centrally managing multiple information processing agents, Means for providing a search engine for searching for and selecting an information processing agent within the information storage unit based on a specific request from the user, A means for setting and executing a communication procedure to coordinate multiple selected information processing agents, A means for providing an external connection unit for users to define, save, and execute their own work procedures, Means for processing and coordinating external request information in order to realize interaction between selected information processing agents, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventional artificial intelligence agents exhibit their performance by specializing in individual tasks. However, when using multiple agents in cooperation, there are problems such as the need for a great deal of time and effort in searching for agents and configuring the cooperation. Also, it has been difficult to efficiently manage communication between agents and flexibly set up a workflow for obtaining the results required by the user himself / herself. Since these problems have been factors hindering business automation and the improvement of personal life, an effective solution has been demanded.

Means for Solving the Problems

[0005] This invention solves the above problems by providing a search engine that centrally manages information on multiple artificial intelligence agents and allows users to search for the optimal agent based on their specific needs. Furthermore, it provides a means for efficient and flexible collaboration among multiple agents by constructing a protocol that enables communication between selected agents and providing an interface that allows users to define and execute their own workflows. This realizes a comprehensive platform that contributes to the automation of corporate operations and the improvement of individuals' lives.

[0006] An "artificial intelligence agent" is a computer program designed to perform specific tasks, equipped with artificial intelligence technology to automate data processing and decision-making.

[0007] A "database" is a system for systematically storing large amounts of information and for efficiently searching and managing it.

[0008] A "search engine" is a program that quickly searches a database for information based on an entered query and provides highly relevant results.

[0009] A "communication protocol" is a standard method that defines the procedures and rules for exchanging data between different computer systems.

[0010] A "user interface" refers to the components, such as operating screens and input devices, that facilitate the smooth exchange of information between a system and a user.

[0011] A "workflow" is a set of steps or procedures that define a sequence of tasks or processes and execute them automatically or semi-automatically. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2]This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0013] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a processor with a reference numeral (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, a RAM (Random Access Memory) with a reference numeral is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, a storage with a reference numeral is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0018] In the following embodiments, a communication I / F (Interface) with a reference numeral is an interface including a communication processor, an antenna, and the like. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] This invention provides a portal system that allows users to efficiently utilize multiple artificial intelligence agents. Specific embodiments thereof are described below.

[0034] First, users can access the portal site and register, search for, select, and link agents. To automate tasks or improve personal life, users first search for an AI agent that suits their specific needs.

[0035] The server receives search queries from users, analyzes agent information in the database, and presents the user with a list of agents that best match the criteria. Detailed information such as each agent's capabilities, pricing, provider information, and available APIs is also provided. This allows users to quickly identify the agent best suited to their needs.

[0036] Next, the necessary processes can be built based on the agent selected by the user. If the user wants to coordinate multiple agents, the server automatically configures the communication protocol between the selected agents, and the APIs between the agents are seamlessly integrated.

[0037] Furthermore, users can design their own workflows through the GUI and save them. This allows users to efficiently automate their daily tasks according to a schedule. For example, by linking a schedule management agent with an email sending agent, users can create a system that automatically sends reminders via email when a meeting is approaching.

[0038] For users who are employees of companies working from home and are managing project progress, linking a progress management agent with a communication agent that facilitates communication among team members makes it possible to achieve efficient project monitoring and information sharing within the team.

[0039] In this way, the portal system according to the present invention provides an environment in which users can easily combine various artificial intelligence agents based on their own requirements and efficiently make the most of their functions.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The user accesses the portal site and enters their authentication information on the login screen. Once the authentication information is entered, the system verifies the user's identity and redirects them to the dashboard.

[0043] Step 2:

[0044] The user enters keywords into the search bar to find an artificial intelligence agent that meets their specific needs.

[0045] Step 3:

[0046] The server receives the search query entered by the user and executes the query in the database. It generates a list of relevant artificial intelligence agents and presents that list to the user.

[0047] Step 4:

[0048] The user selects an agent of interest from the presented agent list and views detailed information. They then review the agent's functions and terms of use and select the agent they wish to use.

[0049] Step 5:

[0050] The user selects multiple agents they wish to use simultaneously and accesses the integration configuration screen.

[0051] Step 6:

[0052] The server automatically generates a communication protocol between the selected agents and configures the connection between APIs based on it. This configuration enables data exchange between agents.

[0053] Step 7:

[0054] Users design their own workflows using the provided GUI. The workflow is formed by specifying the order and trigger conditions for each agent.

[0055] Step 8:

[0056] The terminal executes a saved workflow, sequentially carrying out agent tasks according to user instructions. During task progress, necessary data is shared between agents in real time.

[0057] Step 9:

[0058] The server monitors the task's progress and logs completed steps and errors. For errors, the server takes action such as retrying or suggesting alternative flows.

[0059] Step 10:

[0060] After all tasks are completed, the user reviews the results and decides on the next action. The system's value is maximized by improving workflows or designing new ones based on these results.

[0061] (Example 1)

[0062] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0063] In modern society, the use of intelligent systems plays a crucial role in improving work efficiency and enhancing personal lives. However, technical knowledge and compatibility issues between systems pose obstacles to effectively utilizing and coordinating multiple intelligent systems. Furthermore, accurately understanding the information provided by each intelligent system and finding the optimal combination to suit one's needs is not easy. As a result, users often spend a great deal of time and effort, failing to enjoy the full potential of the systems.

[0064] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0065] In this invention, the server includes means for setting up an information storage device for integrating and managing multiple intelligent systems, means for providing a search device for searching for and selecting intelligent systems within the information storage device based on a specific request from a user, and means for setting up and executing communication control means for coordinating the selected multiple intelligent systems. This makes it possible for users to easily combine and use intelligent systems, construct an optimal processing flow, and realize automation according to their own purposes.

[0066] An "intelligent system" is a technology that has the ability to autonomously process information and propose or implement solutions according to a specific purpose.

[0067] An "information storage device" is a device that centrally manages various types of information related to intelligent systems and allows for quick access to that information as needed.

[0068] A "search device" is a device that provides the function of searching for intelligent systems within an information storage device based on user instructions and selecting an option suitable for the purpose.

[0069] "Communication control means" refers to technology for managing the transmission and reception of data between multiple intelligent systems and for realizing their coordinated operation.

[0070] A "dialogue device" is a device that provides an interface for users to interact with and operate intelligent systems, enabling smooth operation.

[0071] An "operation screen" is a visual user interface that allows users to directly give instructions to the system or input data.

[0072] This system utilizes multiple intelligent systems to support the streamlining of work processes and the automation of personal tasks.

[0073] Users first access the portal site using their device's web browser. This portal site provides an interface for centralized management and collaborative design of intelligent systems.

[0074] The server receives access to the portal site and maintains an information storage device containing information on multiple intelligent systems. Here, the server uses database technology to store information on the functions, provider information, and communication interface information of the intelligent systems. This data is accessed immediately upon user request.

[0075] Users utilize the portal site's search tool to find intelligent systems that meet their needs. By entering keywords and criteria, they can narrow down the search to systems with specific functions and characteristics.

[0076] Based on the search results, the server configures communication control mechanisms to facilitate communication between the specified intelligent systems and integrates the APIs of each system. This process utilizes RESTful APIs and JSON-formatted data exchange.

[0077] Furthermore, users can build their own workflows using a visual interface via an interactive device. For example, by linking a schedule management intelligence system with an email notification intelligence system, it is possible to create a function that automatically sends meeting reminders via email.

[0078] As a concrete example, a possible prompt might be, "I want to create a flow that automatically sends weekly meeting reminder emails using a schedule management agent and an email sending agent." Based on this prompt, the server automatically searches for and configures the relevant intelligent systems to achieve coordination.

[0079] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0080] Step 1:

[0081] The user accesses the portal site using the terminal's web browser. As input, the user enters the portal site's URL into the browser. As output, the portal site's homepage is displayed in the browser. This step provides the user with an entry point to access information about available intelligent systems.

[0082] Step 2:

[0083] Users search for intelligent systems by entering specific keywords or conditions into the search bar on the portal site. As input, users provide search queries based on their needs. The server receives these queries and filters the intelligent systems that match the conditions by searching the database. This process uses SQL queries to retrieve information from the database. As output, a list of search results is displayed to the user.

[0084] Step 3:

[0085] The user selects an intelligent system of interest from the search results. As input, the user clicks on a specific system from the system list. As output, the system's detailed information page is displayed. This step includes reviewing details about each system's capabilities, provider information, and available APIs.

[0086] Step 4:

[0087] When a user wants to connect selected intelligent systems, the server configures the necessary communication protocols to establish communication between these systems. The API information of the selected systems is used as input. The server exchanges information using RESTful APIs and configures the necessary communication settings. The output is data exchange between the systems.

[0088] Step 5:

[0089] Users design their workflows using an interactive device. As input, users determine the order and flow of tasks using a visual drag-and-drop interface. As output, the designed workflow is saved and ready for execution by the system. This step provides an intuitive configuration experience through the user interface.

[0090] Step 6:

[0091] Based on the designed workflow, the server automatically executes tasks and provides feedback to the user. The stored workflow data is used as input. The server executes the specified tasks and generates results. The execution results are notified to the user as output. This step involves the execution of automated processes and verification of their results.

[0092] (Application Example 1)

[0093] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0094] In today's complex information society, citizens and users utilize a variety of information processing agents to improve their lives and work in various ways. However, managing and operating these agents is time-consuming, creating a need for a system that allows for centralized and efficient use of agents. Furthermore, there is a need for an environment that facilitates collaboration and interaction between agents, enabling users to create new value.

[0095] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0096] In this invention, the server includes means for setting up an information storage unit for centrally managing multiple information processing agents; means for providing a search engine for searching for and selecting information processing agents within the information storage unit based on a specific request from a user; means for setting up and executing a communication procedure for linking the selected multiple information processing agents; means for providing an external connection unit for the user to define, save, and execute their own work procedures; and means for processing and coordinating external request information to realize interaction between the selected information processing agents. As a result, the user can improve the efficiency of their life and work by effectively managing and coordinating various information processing agents.

[0097] An "information processing agent" is a type of software that is designed to handle specific tasks and automatically performs various processes in response to user requests.

[0098] An "information storage unit" refers to a storage device or database used to centrally store and manage information and data managed within a system.

[0099] The "search engine" is a software function that searches for information processing agents stored in the information storage unit based on requests from the user and provides the optimal result.

[0100] A "communication procedure" is a set of protocols and rules established to facilitate the smooth transmission and reception of data between selected information processing agents.

[0101] The "external connection section" refers to the user interface and connection mechanism for setting, saving, and executing work procedures and workflows that the user has defined independently.

[0102] "External request information" refers to the data and instructions required to enable interaction between selected information processing agents.

[0103] This invention aims to construct a system that centrally manages and efficiently utilizes information processing agents. The server manages multiple information processing agents using a database built in the information storage unit. The server uses a search engine to quickly find the necessary agents in response to user requests. This engine is implemented in Python, and PostgreSQL is used as the database.

[0104] Communication between selected agents is managed by a communication procedure, using a RESTful API as the communication protocol. This enables seamless data exchange between agents. As a result, users can coordinate multiple agents to efficiently manage their work and daily lives.

[0105] The user terminal is equipped with an external connectivity interface built using React Native. Through this interface, users can define, save, and execute their own work procedures. This system ensures that external request information is properly processed and adjusted, providing optimal services tailored to the user's needs.

[0106] As a concrete example, consider traffic management within a smart city. Users can access a traffic management app from their devices and select a traffic information processing agent to understand real-time traffic conditions and receive suggestions for the optimal route. Furthermore, an energy management app can optimize energy consumption.

[0107] By leveraging the generative AI model, users can use the following prompt: "I want to develop a smart city app. Please suggest a combination of AI agents that will allow users to efficiently manage traffic conditions and energy consumption." Using this prompt, the system can suggest the optimal combination of agents and improve the overall usability of the system.

[0108] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0109] Step 1:

[0110] The user accesses the system from a terminal and enters a search query for the information processing agent. This input includes the user's specific requests and objectives. The server receives this query and starts the search engine.

[0111] Step 2:

[0112] The server searches the database in the information storage unit to find the information processing agent that best matches the user's query. It executes the database query using Python and extracts agent information that meets the conditions. As a result of the data processing, a list of agents is output.

[0113] Step 3:

[0114] The user selects the desired agent from a list of agents displayed on the device. The selection process takes into account the agent's function description and provider information.

[0115] Step 4:

[0116] The server configures communication procedures to establish communication between the agents selected by the user. It uses a RESTful API to construct a protocol for data exchange between the selected agents.

[0117] Step 5:

[0118] Users define their own work procedures using the terminal's external connection port and save them to the system. They design workflows based on their needs and save them for future execution.

[0119] Step 6:

[0120] The server processes and coordinates external request information based on stored workflows, enabling interaction between selected information processing agents. Data calculations provide the optimal response.

[0121] Step 7:

[0122] The output presents the user with the final results. For example, the optimal transportation route or information on optimizing energy consumption might be displayed on the user's terminal. This allows the user to take action based on the suggested solutions.

[0123] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0124] This invention provides a system that recognizes user emotions and optimally utilizes artificial intelligence agents accordingly. This system uses an emotion engine to detect the user's emotional state and uses that information to suggest agents and adjust the workflow.

[0125] Specifically, when a user accesses the portal site, they go through a login process and are then taken to the dashboard. On the dashboard, users can use the search bar to find the artificial intelligence agent they need and view descriptions of each agent's functions, provider information, and API information.

[0126] In this system, the portal incorporates an emotion engine, where the server analyzes user input, environmental data, and past usage history to infer the user's current emotional state. Based on this emotional state, the server dynamically recommends the appropriate agent to the user. This recommendation allows the user to choose the agent that best suits their emotions.

[0127] After selecting agents, the user proceeds to build the workflow. The server automatically configures the communication protocol between the selected agents, ensuring seamless sharing of necessary data. The workflow is freely designed by the user through a GUI, and emotional states detected by the emotion engine are also taken into consideration. This allows for the adaptation and adjustment of the flow according to the user's mood.

[0128] For example, when a user is experiencing stress, recommending an agent specializing in relaxation and support can not only improve work efficiency but also contribute to maintaining mental health. Furthermore, for corporate users managing projects, team member assignments and task reallocations can be made in real time, taking into account the emotional state of team members.

[0129] In this way, the present invention provides a portal system for artificial intelligence agents incorporating an emotion engine, enabling flexible and effective use of agents that take into account the user's emotional state.

[0130] The following describes the processing flow.

[0131] Step 1:

[0132] The user logs into the portal site and accesses the dashboard. Here, the user reviews the system overview and looks at the available features.

[0133] Step 2:

[0134] The server checks the user's login information and past usage history. It connects to the emotion engine and collects initial data to infer the user's recent emotional state.

[0135] Step 3:

[0136] The user uses the search bar to search for a specific artificial intelligence agent. The search query includes keywords related to the user's needs.

[0137] Step 4:

[0138] The server processes the user's search query and queries the database to generate a list of suitable artificial intelligence agents. The list also includes a description of each agent's functions and provider information.

[0139] Step 5:

[0140] Simultaneously, the emotion engine analyzes user input information and environmental data acquired from the microphone and camera to determine the user's emotional state. Based on this, it selects the most suitable agent and generates additional recommendation information.

[0141] Step 6:

[0142] The user reviews the list of agents provided by the server and the recommendations from the sentiment engine, and selects the agent that is best suited to them.

[0143] Step 7:

[0144] The server configures the communication protocol to set up coordination between selected agents. It automatically builds API connections based on the selected agents.

[0145] Step 8:

[0146] Users can navigate to a workflow design screen and adjust the flow based on their emotional state. For example, if a user is feeling stressed, the flow can incorporate relaxing tasks or agents that provide enhanced support.

[0147] Step 9:

[0148] The device executes a workflow designed by the user. During execution, the emotion engine monitors the emotional state in real time and dynamically adjusts the flow as needed.

[0149] Step 10:

[0150] The server monitors the progress of tasks and the status of agents, and logs completed tasks and their results. If errors occur, appropriate error handling is performed.

[0151] Step 11:

[0152] After all processes are complete, the user reviews the results and makes final decisions and determines the next actions. Based on these results, the user can improve or save the new workflow.

[0153] (Example 2)

[0154] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0155] In recent years, technologies that utilize multiple intelligent agents to streamline operations have advanced. However, conventional systems do not adequately support the selection of appropriate agents and adjustment of work processes based on the user's emotional state. Therefore, there is a need for systems that can flexibly respond to user requests and feelings.

[0156] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0157] In this invention, the server includes means for setting up an information infrastructure for centrally managing information on multiple intelligent agents, means for providing a search function for searching for and selecting intelligent agents within the information infrastructure based on a specific request from a user, and means for analyzing the user's emotional state and recommending the intelligent agents based on the analysis results. This makes it possible to select an appropriate agent according to the user's emotional state and to adapt and adjust the work process accordingly.

[0158] An "intelligent agent" is a software program that performs specific tasks and responds or performs actions according to the user's requests.

[0159] An "information infrastructure" is a system structure for integrating and managing data from multiple intelligent agents.

[0160] A "user" is a person or organization that uses this system to search for intelligent agents or build workflows.

[0161] The "search function" is a means of searching for and selecting intelligent agents within the information infrastructure based on user requests.

[0162] "Emotional state" refers to information that indicates the user's psychological or emotional state, and is analyzed by the system.

[0163] A "business process" refers to a series of tasks or processes that a user performs using an intelligent agent.

[0164] An "information exchange method" is a protocol or procedure for enabling data communication and collaborative work between multiple intelligent agents.

[0165] This invention constructs a system that utilizes intelligent agents to provide the optimal approach based on the user's emotional state. A server functions as the core of this system, building an information infrastructure that manages information about the intelligent agents. This infrastructure stores the characteristics and functions of each agent. Furthermore, the server uses a search function to select an agent according to the user's specific request. It also uses a generative AI model for emotion analysis to infer the user's emotional state and recommends the most suitable agent based on this information.

[0166] The terminal provides an interface for users to operate this system. On the terminal, users can search for agents and define work processes. Furthermore, the terminal presents the user with agent information obtained from the information infrastructure, assisting in their selection. The selected agents are seamlessly integrated through an information exchange method configured by the server.

[0167] Users can design their own work processes and adjust them as needed through the provided graphical user interface. During this process, advice and task reorganization tailored to their emotional state are provided in real time.

[0168] For example, if a user feels the need to relax, the server will perform emotion analysis and recommend an agent specializing in relaxation. Furthermore, a prompt could be a question posed to the generating AI model such as, "What is the most effective agent action when the user is feeling anxious?"

[0169] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0170] Step 1:

[0171] The server receives login information when a user accesses the portal site. The server accesses the database and verifies the entered user ID and password. In this step, the user ID and password are used as input, and the user's authentication status (authentication successful / failed) is generated as output. If authentication is successful, the dashboard screen is sent to the terminal.

[0172] Step 2:

[0173] The terminal displays a dashboard screen to the user. Here, the user uses the search bar to find the necessary intelligent agent. The keywords entered by the user are used as input data, and a search query is sent from the terminal to the server. The server uses this query to search the agent information infrastructure, generates a list of relevant agents, and outputs it. The terminal then presents this list to the user.

[0174] Step 3:

[0175] The server uses an emotion engine to analyze the user's emotional state. Inputs include the user's mouse and keyboard inputs, as well as environmental data. A generative AI model processes this data to estimate the user's emotional state. The output is the estimated emotional state information. Based on this information, the server dynamically selects the most suitable agent for the user and sends it to the terminal as a recommendation list.

[0176] Step 4:

[0177] The user selects their desired agent from a recommended agent list on their terminal. The selected agent information is sent from the terminal to the server. The server automatically configures the appropriate information exchange method to enable cooperation between the selected agents. In this system, the selected agent information is used as input, and communication protocol configuration data is generated as output.

[0178] Step 5:

[0179] Users design work processes using the terminal's graphical user interface. The server adapts and adjusts the work processes according to the user's emotional state. Using the designed work process data and emotional state data as input, an adjusted work process is generated as output. The work process is optimized in real time and displayed to the user.

[0180] (Application Example 2)

[0181] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0182] In today's information society, there is a demand for flexible and effective use of artificial intelligence that responds to users' emotions and states, but conventional solutions have been insufficient. The challenge lies in appropriately analyzing users' emotions and realizing dynamic recommendations of intelligent functions and optimization of work procedures based on that analysis.

[0183] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0184] In this invention, the server includes means for setting up an information storage device for centrally managing multiple intelligent functions; means for providing a search mechanism for searching for and selecting intelligent functions within the information storage device based on a specific request from a user; means for setting up and executing a communication method for coordinating the selected multiple intelligent functions; and means for providing an emotion analysis mechanism for analyzing the user's emotional state and dynamically recommending intelligent functions accordingly. This enables the flexible and effective use of artificial intelligence in accordance with the user's emotions.

[0185] "Intelligent function" refers to artificial intelligence systems or programs designed to perform specific tasks.

[0186] An "information storage device" refers to a system for efficiently storing and managing data.

[0187] A "search mechanism" refers to a means of searching for and selecting necessary information and functions.

[0188] "Communication method" refers to the protocols and procedures that enable multiple functions or systems to cooperate.

[0189] The "interactive interface" refers to the interface that users use to give instructions to the system or to retrieve information.

[0190] "Emotional state" refers to the emotional state or mental condition that an individual is experiencing.

[0191] An "emotion analysis mechanism" refers to a system or algorithm for analyzing and understanding a user's emotional state.

[0192] "Dynamic recommendation" refers to presenting appropriate options in real time based on the user's state and circumstances.

[0193] The system implementing the present invention consists of a series of programs for analyzing the user's emotional state and dynamically providing intelligent functions based on that analysis.

[0194] The server centrally manages multiple intelligent functions using an information storage device. This makes the intelligent functions of the entire system easily accessible. Based on a user's specific request, the server uses a search mechanism to search for and select the appropriate intelligent function within the information storage device. In this process, the most effective intelligent function is selected.

[0195] The emotion analysis mechanism processes voice and image data input by the user. Specifically, it uses software such as Python, OpenCV, and TENSORFLOW® to perform real-time emotion analysis. Based on this data analysis, the server aggregates the emotional state and dynamically recommends appropriate intelligence functions. For example, if it determines that the user is feeling stressed, it selects an intelligence function specifically for relaxation.

[0196] The device interacts with the user through a user interface. It assists the user in selecting emotionally appropriate intelligent functions and defining their own work procedures, and effectively coordinates these intelligent functions through communication methods.

[0197] For example, when a user inputs information through a home robot, they can give instructions such as, "Analyze the user's emotions and suggest a word of encouragement." Based on this prompt, the system will begin the analysis and provide appropriate intelligent functions.

[0198] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0199] Step 1:

[0200] The server receives input data from the user. At this stage, input, as well as voice and image data, are acquired through the terminal. Then, the server aggregates the necessary data, taking into account the user's profile, history, etc.

[0201] Step 2:

[0202] The server passes the aggregated input data to the emotion analysis mechanism. This mechanism uses Python, OpenCV, and TensorFlow to analyze the user's emotional state. Specifically, it extracts characteristic patterns from audio data and facial image data, and generates emotion labels (e.g., stress, joy, etc.) from these. The analysis results are used for subsequent processing.

[0203] Step 3:

[0204] Based on the analysis results, the server selects the appropriate intelligent function from within the information storage device. At this stage, a search mechanism searches for the intelligent function that best matches the analyzed emotion label. Then, it collects information about that intelligent function and prepares to set up the communication method.

[0205] Step 4:

[0206] The server provides the terminal with detailed information about the selected intelligent functions. This includes descriptions of the available functions and instructions on how to configure them. Based on this information, the user makes a final decision on which intelligent functions to use through the interactive interface on the terminal.

[0207] Step 5:

[0208] The terminal completes preparations to execute the specified intelligent function based on the user's selection. It sets the communication method and sends commands to ensure seamless coordination of the intelligent functions. Then, once ready, it actually executes the intelligent function and provides the results to the user.

[0209] Step 6:

[0210] After receiving the results of the performed intelligent function, the user inputs feedback into the terminal. This feedback is sent to the server as data for future processing improvements and further sentiment analysis, and is used to train the system.

[0211] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0212] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0213] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0214] [Second Embodiment]

[0215] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0216] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0217] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0218] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0219] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0220] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0221] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0222] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0223] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0224] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0225] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0226] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0227] This invention provides a portal system that allows users to efficiently utilize multiple artificial intelligence agents. Specific embodiments thereof are described below.

[0228] First, users can access the portal site and register, search for, select, and link agents. To automate tasks or improve personal life, users first search for an AI agent that suits their specific needs.

[0229] The server receives search queries from users, analyzes agent information in the database, and presents the user with a list of agents that best match the criteria. Detailed information such as each agent's capabilities, pricing, provider information, and available APIs is also provided. This allows users to quickly identify the agent best suited to their needs.

[0230] Next, the necessary processes can be built based on the agent selected by the user. If the user wants to coordinate multiple agents, the server automatically configures the communication protocol between the selected agents, and the APIs between the agents are seamlessly integrated.

[0231] Furthermore, users can design their own workflows through the GUI and save them. This allows users to efficiently automate their daily tasks according to a schedule. For example, by linking a schedule management agent with an email sending agent, users can create a system that automatically sends reminders via email when a meeting is approaching.

[0232] For users who are employees of companies working from home and are managing project progress, linking a progress management agent with a communication agent that facilitates communication among team members makes it possible to achieve efficient project monitoring and information sharing within the team.

[0233] In this way, the portal system according to the present invention provides an environment in which users can easily combine various artificial intelligence agents based on their own requirements and efficiently make the most of their functions.

[0234] The following describes the processing flow.

[0235] Step 1:

[0236] The user accesses the portal site and enters their authentication information on the login screen. Once the authentication information is entered, the system verifies the user's identity and redirects them to the dashboard.

[0237] Step 2:

[0238] The user enters keywords into the search bar to find an artificial intelligence agent that meets their specific needs.

[0239] Step 3:

[0240] The server receives the search query entered by the user and executes the query in the database. It generates a list of relevant artificial intelligence agents and presents that list to the user.

[0241] Step 4:

[0242] The user selects an agent of interest from the presented agent list and views detailed information. They then review the agent's functions and terms of use and select the agent they wish to use.

[0243] Step 5:

[0244] The user selects multiple agents they wish to use simultaneously and accesses the integration configuration screen.

[0245] Step 6:

[0246] The server automatically generates a communication protocol between the selected agents and configures the connection between APIs based on it. This configuration enables data exchange between agents.

[0247] Step 7:

[0248] Users design their own workflows using the provided GUI. The workflow is formed by specifying the order and trigger conditions for each agent.

[0249] Step 8:

[0250] The terminal executes a saved workflow, sequentially carrying out agent tasks according to user instructions. During task progress, necessary data is shared between agents in real time.

[0251] Step 9:

[0252] The server monitors the task's progress and logs completed steps and errors. For errors, the server takes action such as retrying or suggesting alternative flows.

[0253] Step 10:

[0254] After all tasks are completed, the user reviews the results and decides on the next action. The system's value is maximized by improving workflows or designing new ones based on these results.

[0255] (Example 1)

[0256] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0257] In modern society, the use of intelligent systems plays a crucial role in improving work efficiency and enhancing personal lives. However, technical knowledge and compatibility issues between systems pose obstacles to effectively utilizing and coordinating multiple intelligent systems. Furthermore, accurately understanding the information provided by each intelligent system and finding the optimal combination to suit one's needs is not easy. As a result, users often spend a great deal of time and effort, failing to enjoy the full potential of the systems.

[0258] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0259] In this invention, the server includes means for setting up an information storage device for integrating and managing multiple intelligent systems, means for providing a search device for searching for and selecting intelligent systems within the information storage device based on a specific request from a user, and means for setting up and executing communication control means for coordinating the selected multiple intelligent systems. This makes it possible for users to easily combine and use intelligent systems, construct an optimal processing flow, and realize automation according to their own purposes.

[0260] An "intelligent system" is a technology that has the ability to autonomously process information and propose or implement solutions according to a specific purpose.

[0261] An "information storage device" is a device that centrally manages various types of information related to intelligent systems and allows for quick access to that information as needed.

[0262] A "search device" is a device that provides the function of searching for intelligent systems within an information storage device based on user instructions and selecting an option suitable for the purpose.

[0263] "Communication control means" refers to technology for managing the transmission and reception of data between multiple intelligent systems and for realizing their coordinated operation.

[0264] A "dialogue device" is a device that provides an interface for users to interact with and operate intelligent systems, enabling smooth operation.

[0265] An "operation screen" is a visual user interface that allows users to directly give instructions to the system or input data.

[0266] This system utilizes multiple intelligent systems to support the streamlining of work processes and the automation of personal tasks.

[0267] Users first access the portal site using their device's web browser. This portal site provides an interface for centralized management and collaborative design of intelligent systems.

[0268] The server receives access to the portal site and maintains an information storage device containing information on multiple intelligent systems. Here, the server uses database technology to store information on the functions, provider information, and communication interface information of the intelligent systems. This data is accessed immediately upon user request.

[0269] Users utilize the portal site's search tool to find intelligent systems that meet their needs. By entering keywords and criteria, they can narrow down the search to systems with specific functions and characteristics.

[0270] Based on the search results, the server configures communication control mechanisms to facilitate communication between the specified intelligent systems and integrates the APIs of each system. This process utilizes RESTful APIs and JSON-formatted data exchange.

[0271] Furthermore, users can build their own workflows using a visual interface via an interactive device. For example, by linking a schedule management intelligence system with an email notification intelligence system, it is possible to create a function that automatically sends meeting reminders via email.

[0272] As a concrete example, a possible prompt might be, "I want to create a flow that automatically sends weekly meeting reminder emails using a schedule management agent and an email sending agent." Based on this prompt, the server automatically searches for and configures the relevant intelligent systems to achieve coordination.

[0273] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0274] Step 1:

[0275] The user accesses the portal site using the terminal's web browser. As input, the user enters the portal site's URL into the browser. As output, the portal site's homepage is displayed in the browser. This step provides the user with an entry point to access information about available intelligent systems.

[0276] Step 2:

[0277] Users search for intelligent systems by entering specific keywords or conditions into the search bar on the portal site. As input, users provide search queries based on their needs. The server receives these queries and filters the intelligent systems that match the conditions by searching the database. This process uses SQL queries to retrieve information from the database. As output, a list of search results is displayed to the user.

[0278] Step 3:

[0279] The user selects an intelligent system of interest from the search results. As input, the user clicks on a specific system from the system list. As output, the system's detailed information page is displayed. This step includes reviewing details about each system's capabilities, provider information, and available APIs.

[0280] Step 4:

[0281] When a user wants to connect selected intelligent systems, the server configures the necessary communication protocols to establish communication between these systems. The API information of the selected systems is used as input. The server exchanges information using RESTful APIs and configures the necessary communication settings. The output is data exchange between the systems.

[0282] Step 5:

[0283] The user designs their own workflow using an interactive device. As input, the user uses a visual drag-and-drop interface to determine the order and flow of tasks. As output, the designed workflow is saved and made executable by the system. In this step, an operation for intuitively making settings through the user interface is provided.

[0284] Step 6:

[0285] Based on the designed workflow, the server automatically executes tasks and provides feedback to the user on the results. As input, the data of the saved workflow is used. The server executes the specified tasks and generates results. As output, the execution results are notified to the user. This step includes the execution of automated processing and the confirmation of its results.

[0286] (Application Example 1)

[0287] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0288] In modern complex information societies, citizens and users utilize various information processing agents to seek various improvements in their lives and work. However, managing and operating these agents requires a great deal of effort, so a mechanism that enables the agents to be utilized in a unified and efficient manner is demanded. Furthermore, an environment that facilitates cooperation and interaction between agents and allows users to create new value is needed.

[0289] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0290] In this invention, the server includes means for setting up an information storage unit for centrally managing multiple information processing agents; means for providing a search engine for searching for and selecting information processing agents within the information storage unit based on a specific request from a user; means for setting up and executing a communication procedure for linking the selected multiple information processing agents; means for providing an external connection unit for the user to define, save, and execute their own work procedures; and means for processing and coordinating external request information to realize interaction between the selected information processing agents. As a result, the user can improve the efficiency of their life and work by effectively managing and coordinating various information processing agents.

[0291] An "information processing agent" is a type of software that is designed to handle specific tasks and automatically performs various processes in response to user requests.

[0292] An "information storage unit" refers to a storage device or database used to centrally store and manage information and data managed within a system.

[0293] The "search engine" is a software function that searches for information processing agents stored in the information storage unit based on requests from the user and provides the optimal result.

[0294] A "communication procedure" is a set of protocols and rules established to facilitate the smooth transmission and reception of data between selected information processing agents.

[0295] The "external connection section" refers to the user interface and connection mechanism for setting, saving, and executing work procedures and workflows that the user has defined independently.

[0296] "External request information" refers to the data and instructions required to enable interaction between selected information processing agents.

[0297] This invention aims to construct a system that centrally manages and efficiently utilizes information processing agents. The server manages multiple information processing agents using a database built in the information storage unit. The server uses a search engine to quickly find the necessary agents in response to user requests. This engine is implemented in Python, and PostgreSQL is used as the database.

[0298] Communication between selected agents is managed by a communication procedure, using a RESTful API as the communication protocol. This enables seamless data exchange between agents. As a result, users can coordinate multiple agents to efficiently manage their work and daily lives.

[0299] The user terminal is equipped with an external connectivity interface built using React Native. Through this interface, users can define, save, and execute their own work procedures. This system ensures that external request information is properly processed and adjusted, providing optimal services tailored to the user's needs.

[0300] As a concrete example, consider traffic management within a smart city. Users can access a traffic management app from their devices and select a traffic information processing agent to understand real-time traffic conditions and receive suggestions for the optimal route. Furthermore, an energy management app can optimize energy consumption.

[0301] By leveraging the generative AI model, users can use the following prompt: "I want to develop a smart city app. Please suggest a combination of AI agents that will allow users to efficiently manage traffic conditions and energy consumption." Using this prompt, the system can suggest the optimal combination of agents and improve the overall usability of the system.

[0302] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0303] Step 1:

[0304] The user accesses the system from a terminal and inputs a search query for an information processing agent. This input includes the user's specific requirements and purposes. The server receives this query and activates the search engine.

[0305] Step 2:

[0306] The server searches the database in the information storage unit to search for the information processing agent that best matches the user's query. It executes a database query using Python and extracts agent information that meets the conditions. As a result of data processing, a list of agents is output.

[0307] Step 3:

[0308] The user selects the necessary agent from the list of agents displayed on the terminal. The function description and provider information of the agent are considered for the selection.

[0309] Step 4:

[0310] The server sets up the communication procedure to establish communication between the agents selected by the user. It constructs a protocol for data exchange between the selected agents using the RESTful API.

[0311] Step 5:

[0312] The user defines their own working procedure using the external connection part of the terminal and saves it in the system. A workflow based on the user's needs is designed and saved for future execution.

[0313] Step 6:

[0314] The server processes and coordinates external request information based on stored workflows, enabling interaction between selected information processing agents. Data calculations provide the optimal response.

[0315] Step 7:

[0316] The output presents the user with the final results. For example, the optimal transportation route or information on optimizing energy consumption might be displayed on the user's terminal. This allows the user to take action based on the suggested solutions.

[0317] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0318] This invention provides a system that recognizes user emotions and optimally utilizes artificial intelligence agents accordingly. This system uses an emotion engine to detect the user's emotional state and uses that information to suggest agents and adjust the workflow.

[0319] Specifically, when a user accesses the portal site, they go through a login process and are then taken to the dashboard. On the dashboard, users can use the search bar to find the artificial intelligence agent they need and view descriptions of each agent's functions, provider information, and API information.

[0320] In this system, the portal incorporates an emotion engine, where the server analyzes user input, environmental data, and past usage history to infer the user's current emotional state. Based on this emotional state, the server dynamically recommends the appropriate agent to the user. This recommendation allows the user to choose the agent that best suits their emotions.

[0321] After selecting agents, the user proceeds to build the workflow. The server automatically configures the communication protocol between the selected agents, ensuring seamless sharing of necessary data. The workflow is freely designed by the user through a GUI, and emotional states detected by the emotion engine are also taken into consideration. This allows for the adaptation and adjustment of the flow according to the user's mood.

[0322] For example, when a user is experiencing stress, recommending an agent specializing in relaxation and support can not only improve work efficiency but also contribute to maintaining mental health. Furthermore, for corporate users managing projects, team member assignments and task reallocations can be made in real time, taking into account the emotional state of team members.

[0323] In this way, the present invention provides a portal system for artificial intelligence agents incorporating an emotion engine, enabling flexible and effective use of agents that take into account the user's emotional state.

[0324] The following describes the processing flow.

[0325] Step 1:

[0326] The user logs into the portal site and accesses the dashboard. Here, the user reviews the system overview and looks at the available features.

[0327] Step 2:

[0328] The server checks the user's login information and past usage history. It connects to the emotion engine and collects initial data to infer the user's recent emotional state.

[0329] Step 3:

[0330] The user uses the search bar to search for a specific artificial intelligence agent. The search query includes keywords related to the user's needs.

[0331] Step 4:

[0332] The server processes the user's search query and queries the database to generate a list of suitable artificial intelligence agents. The list also includes a description of each agent's functions and provider information.

[0333] Step 5:

[0334] Simultaneously, the emotion engine analyzes user input information and environmental data acquired from the microphone and camera to determine the user's emotional state. Based on this, it selects the most suitable agent and generates additional recommendation information.

[0335] Step 6:

[0336] The user reviews the list of agents provided by the server and the recommendations from the sentiment engine, and selects the agent that is best suited to them.

[0337] Step 7:

[0338] The server configures the communication protocol to set up coordination between selected agents. It automatically builds API connections based on the selected agents.

[0339] Step 8:

[0340] Users can navigate to a workflow design screen and adjust the flow based on their emotional state. For example, if a user is feeling stressed, the flow can incorporate relaxing tasks or agents that provide enhanced support.

[0341] Step 9:

[0342] The device executes a workflow designed by the user. During execution, the emotion engine monitors the emotional state in real time and dynamically adjusts the flow as needed.

[0343] Step 10:

[0344] The server monitors the progress of tasks and the status of agents, and logs completed tasks and their results. If errors occur, appropriate error handling is performed.

[0345] Step 11:

[0346] After all processes are complete, the user reviews the results and makes final decisions and determines the next actions. Based on these results, the user can improve or save the new workflow.

[0347] (Example 2)

[0348] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0349] In recent years, technologies that utilize multiple intelligent agents to streamline operations have advanced. However, conventional systems do not adequately support the selection of appropriate agents and adjustment of work processes based on the user's emotional state. Therefore, there is a need for systems that can flexibly respond to user requests and feelings.

[0350] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0351] In this invention, the server includes means for setting up an information infrastructure for centrally managing information on multiple intelligent agents, means for providing a search function for searching for and selecting intelligent agents within the information infrastructure based on a specific request from a user, and means for analyzing the user's emotional state and recommending the intelligent agents based on the analysis results. This makes it possible to select an appropriate agent according to the user's emotional state and to adapt and adjust the work process accordingly.

[0352] An "intelligent agent" is a software program that performs specific tasks and responds or performs actions according to the user's requests.

[0353] An "information infrastructure" is a system structure for integrating and managing data from multiple intelligent agents.

[0354] A "user" is a person or organization that uses this system to search for intelligent agents or build workflows.

[0355] The "search function" is a means of searching for and selecting intelligent agents within the information infrastructure based on user requests.

[0356] "Emotional state" refers to information that indicates the user's psychological or emotional state, and is analyzed by the system.

[0357] A "business process" refers to a series of tasks or processes that a user performs using an intelligent agent.

[0358] An "information exchange method" is a protocol or procedure for enabling data communication and collaborative work between multiple intelligent agents.

[0359] This invention constructs a system that utilizes intelligent agents to provide the optimal approach based on the user's emotional state. A server functions as the core of this system, building an information infrastructure that manages information about the intelligent agents. This infrastructure stores the characteristics and functions of each agent. Furthermore, the server uses a search function to select an agent according to the user's specific request. It also uses a generative AI model for emotion analysis to infer the user's emotional state and recommends the most suitable agent based on this information.

[0360] The terminal provides an interface for users to operate this system. On the terminal, users can search for agents and define work processes. Furthermore, the terminal presents the user with agent information obtained from the information infrastructure, assisting in their selection. The selected agents are seamlessly integrated through an information exchange method configured by the server.

[0361] Users can design their own work processes and adjust them as needed through the provided graphical user interface. During this process, advice and task reorganization tailored to their emotional state are provided in real time.

[0362] For example, if a user feels the need to relax, the server will perform emotion analysis and recommend an agent specializing in relaxation. Furthermore, a prompt could be a question posed to the generating AI model such as, "What is the most effective agent action when the user is feeling anxious?"

[0363] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0364] Step 1:

[0365] The server receives login information when a user accesses the portal site. The server accesses the database and verifies the entered user ID and password. In this step, the user ID and password are used as input, and the user's authentication status (authentication successful / failed) is generated as output. If authentication is successful, the dashboard screen is sent to the terminal.

[0366] Step 2:

[0367] The terminal displays a dashboard screen to the user. Here, the user uses the search bar to find the necessary intelligent agent. The keywords entered by the user are used as input data, and a search query is sent from the terminal to the server. The server uses this query to search the agent information infrastructure, generates a list of relevant agents, and outputs it. The terminal then presents this list to the user.

[0368] Step 3:

[0369] The server uses an emotion engine to analyze the user's emotional state. Inputs include the user's mouse and keyboard inputs, as well as environmental data. A generative AI model processes this data to estimate the user's emotional state. The output is the estimated emotional state information. Based on this information, the server dynamically selects the most suitable agent for the user and sends it to the terminal as a recommendation list.

[0370] Step 4:

[0371] The user selects their desired agent from a recommended agent list on their terminal. The selected agent information is sent from the terminal to the server. The server automatically configures the appropriate information exchange method to enable cooperation between the selected agents. In this system, the selected agent information is used as input, and communication protocol configuration data is generated as output.

[0372] Step 5:

[0373] Users design work processes using the terminal's graphical user interface. The server adapts and adjusts the work processes according to the user's emotional state. Using the designed work process data and emotional state data as input, an adjusted work process is generated as output. The work process is optimized in real time and displayed to the user.

[0374] (Application Example 2)

[0375] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0376] In today's information society, there is a demand for flexible and effective use of artificial intelligence that responds to users' emotions and states, but conventional solutions have been insufficient. The challenge lies in appropriately analyzing users' emotions and realizing dynamic recommendations of intelligent functions and optimization of work procedures based on that analysis.

[0377] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0378] In this invention, the server includes means for setting up an information storage device for centrally managing multiple intelligent functions; means for providing a search mechanism for searching for and selecting intelligent functions within the information storage device based on a specific request from a user; means for setting up and executing a communication method for coordinating the selected multiple intelligent functions; and means for providing an emotion analysis mechanism for analyzing the user's emotional state and dynamically recommending intelligent functions accordingly. This enables the flexible and effective use of artificial intelligence in accordance with the user's emotions.

[0379] "Intelligent function" refers to artificial intelligence systems or programs designed to perform specific tasks.

[0380] An "information storage device" refers to a system for efficiently storing and managing data.

[0381] A "search mechanism" refers to a means of searching for and selecting necessary information and functions.

[0382] "Communication method" refers to the protocols and procedures that enable multiple functions or systems to cooperate.

[0383] The "interactive interface" refers to the interface that users use to give instructions to the system or to retrieve information.

[0384] "Emotional state" refers to the emotional state or mental condition that an individual is experiencing.

[0385] An "emotion analysis mechanism" refers to a system or algorithm for analyzing and understanding a user's emotional state.

[0386] "Dynamic recommendation" refers to presenting appropriate options in real time based on the user's state and circumstances.

[0387] The system implementing the present invention consists of a series of programs for analyzing the user's emotional state and dynamically providing intelligent functions based on that analysis.

[0388] The server centrally manages multiple intelligent functions using an information storage device. This makes the intelligent functions of the entire system easily accessible. Based on a user's specific request, the server uses a search mechanism to search for and select the appropriate intelligent function within the information storage device. In this process, the most effective intelligent function is selected.

[0389] The emotion analysis mechanism processes the voice and image data input by the user. Specifically, it uses software such as Python, OpenCV, and TensorFlow to perform real-time emotion analysis. Based on this data analysis, the server aggregates the emotional state and dynamically recommends appropriate intelligence functions. For example, if it determines that the user is feeling stressed, it selects an intelligence function specifically for relaxation.

[0390] The device interacts with the user through a user interface. It assists the user in selecting emotionally appropriate intelligent functions and defining their own work procedures, and effectively coordinates these intelligent functions through communication methods.

[0391] For example, when a user inputs information through a home robot, they can give instructions such as, "Analyze the user's emotions and suggest a word of encouragement." Based on this prompt, the system will begin the analysis and provide appropriate intelligent functions.

[0392] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0393] Step 1:

[0394] The server receives input data from the user. At this stage, input, as well as voice and image data, are acquired through the terminal. Then, the server aggregates the necessary data, taking into account the user's profile, history, etc.

[0395] Step 2:

[0396] The server passes the aggregated input data to the emotion analysis mechanism. This mechanism uses Python, OpenCV, and TensorFlow to analyze the user's emotional state. Specifically, it extracts characteristic patterns from audio data and facial image data, and generates emotion labels (e.g., stress, joy, etc.) from these. The analysis results are used for subsequent processing.

[0397] Step 3:

[0398] Based on the analysis results, the server selects the appropriate intelligent function from within the information storage device. At this stage, a search mechanism searches for the intelligent function that best matches the analyzed emotion label. Then, it collects information about that intelligent function and prepares to set up the communication method.

[0399] Step 4:

[0400] The server provides the terminal with detailed information about the selected intelligent functions. This includes descriptions of the available functions and instructions on how to configure them. Based on this information, the user makes a final decision on which intelligent functions to use through the interactive interface on the terminal.

[0401] Step 5:

[0402] The terminal completes preparations to execute the specified intelligent function based on the user's selection. It sets the communication method and sends commands to ensure seamless coordination of the intelligent functions. Then, once ready, it actually executes the intelligent function and provides the results to the user.

[0403] Step 6:

[0404] After receiving the results of the performed intelligent function, the user inputs feedback into the terminal. This feedback is sent to the server as data for future processing improvements and further sentiment analysis, and is used to train the system.

[0405] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0406] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0407] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0408] [Third Embodiment]

[0409] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0410] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0411] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0412] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0413] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0414] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0415] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0416] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0417] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0418] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0419] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0420] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0421] This invention provides a portal system that allows users to efficiently utilize multiple artificial intelligence agents. Specific embodiments thereof are described below.

[0422] First, users can access the portal site and register, search for, select, and link agents. To automate tasks or improve personal life, users first search for an AI agent that suits their specific needs.

[0423] The server receives search queries from users, analyzes agent information in the database, and presents the user with a list of agents that best match the criteria. Detailed information such as each agent's capabilities, pricing, provider information, and available APIs is also provided. This allows users to quickly identify the agent best suited to their needs.

[0424] Next, the necessary processes can be built based on the agent selected by the user. If the user wants to coordinate multiple agents, the server automatically configures the communication protocol between the selected agents, and the APIs between the agents are seamlessly integrated.

[0425] Furthermore, users can design their own workflows through the GUI and save them. This allows users to efficiently automate their daily tasks according to a schedule. For example, by linking a schedule management agent with an email sending agent, users can create a system that automatically sends reminders via email when a meeting is approaching.

[0426] For users who are employees of companies working from home and are managing project progress, linking a progress management agent with a communication agent that facilitates communication among team members makes it possible to achieve efficient project monitoring and information sharing within the team.

[0427] In this way, the portal system according to the present invention provides an environment in which users can easily combine various artificial intelligence agents based on their own requirements and efficiently make the most of their functions.

[0428] The following describes the processing flow.

[0429] Step 1:

[0430] The user accesses the portal site and enters their authentication information on the login screen. Once the authentication information is entered, the system verifies the user's identity and redirects them to the dashboard.

[0431] Step 2:

[0432] The user enters keywords into the search bar to find an artificial intelligence agent that meets their specific needs.

[0433] Step 3:

[0434] The server receives the search query entered by the user and executes the query in the database. It generates a list of relevant artificial intelligence agents and presents that list to the user.

[0435] Step 4:

[0436] The user selects an agent of interest from the presented agent list and views detailed information. They then review the agent's functions and terms of use and select the agent they wish to use.

[0437] Step 5:

[0438] The user selects multiple agents they wish to use simultaneously and accesses the integration configuration screen.

[0439] Step 6:

[0440] The server automatically generates a communication protocol between the selected agents and configures the connection between APIs based on it. This configuration enables data exchange between agents.

[0441] Step 7:

[0442] Users design their own workflows using the provided GUI. The workflow is formed by specifying the order and trigger conditions for each agent.

[0443] Step 8:

[0444] The terminal executes a saved workflow, sequentially carrying out agent tasks according to user instructions. During task progress, necessary data is shared between agents in real time.

[0445] Step 9:

[0446] The server monitors the task's progress and logs completed steps and errors. For errors, the server takes action such as retrying or suggesting alternative flows.

[0447] Step 10:

[0448] After all tasks are completed, the user reviews the results and decides on the next action. The system's value is maximized by improving workflows or designing new ones based on these results.

[0449] (Example 1)

[0450] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0451] In modern society, the use of intelligent systems plays a crucial role in improving work efficiency and enhancing personal lives. However, technical knowledge and compatibility issues between systems pose obstacles to effectively utilizing and coordinating multiple intelligent systems. Furthermore, accurately understanding the information provided by each intelligent system and finding the optimal combination to suit one's needs is not easy. As a result, users often spend a great deal of time and effort, failing to enjoy the full potential of the systems.

[0452] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0453] In this invention, the server includes means for setting up an information storage device for integrating and managing multiple intelligent systems, means for providing a search device for searching for and selecting intelligent systems within the information storage device based on a specific request from a user, and means for setting up and executing communication control means for coordinating the selected multiple intelligent systems. This makes it possible for users to easily combine and use intelligent systems, construct an optimal processing flow, and realize automation according to their own purposes.

[0454] An "intelligent system" is a technology that has the ability to autonomously process information and propose or implement solutions according to a specific purpose.

[0455] An "information storage device" is a device that centrally manages various types of information related to intelligent systems and allows for quick access to that information as needed.

[0456] A "search device" is a device that provides the function of searching for intelligent systems within an information storage device based on user instructions and selecting an option suitable for the purpose.

[0457] "Communication control means" refers to technology for managing the transmission and reception of data between multiple intelligent systems and for realizing their coordinated operation.

[0458] A "dialogue device" is a device that provides an interface for users to interact with and operate intelligent systems, enabling smooth operation.

[0459] An "operation screen" is a visual user interface that allows users to directly give instructions to the system or input data.

[0460] This system utilizes multiple intelligent systems to support the streamlining of work processes and the automation of personal tasks.

[0461] Users first access the portal site using their device's web browser. This portal site provides an interface for centralized management and collaborative design of intelligent systems.

[0462] The server receives access to the portal site and maintains an information storage device containing information on multiple intelligent systems. Here, the server uses database technology to store information on the functions, provider information, and communication interface information of the intelligent systems. This data is accessed immediately upon user request.

[0463] Users utilize the portal site's search tool to find intelligent systems that meet their needs. By entering keywords and criteria, they can narrow down the search to systems with specific functions and characteristics.

[0464] Based on the search results, the server configures communication control mechanisms to facilitate communication between the specified intelligent systems and integrates the APIs of each system. This process utilizes RESTful APIs and JSON-formatted data exchange.

[0465] Furthermore, users can build their own workflows using a visual interface via an interactive device. For example, by linking a schedule management intelligence system with an email notification intelligence system, it is possible to create a function that automatically sends meeting reminders via email.

[0466] As a concrete example, a possible prompt might be, "I want to create a flow that automatically sends weekly meeting reminder emails using a schedule management agent and an email sending agent." Based on this prompt, the server automatically searches for and configures the relevant intelligent systems to achieve coordination.

[0467] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0468] Step 1:

[0469] The user accesses the portal site using the terminal's web browser. As input, the user enters the portal site's URL into the browser. As output, the portal site's homepage is displayed in the browser. This step provides the user with an entry point to access information about available intelligent systems.

[0470] Step 2:

[0471] Users search for intelligent systems by entering specific keywords or conditions into the search bar on the portal site. As input, users provide search queries based on their needs. The server receives these queries and filters the intelligent systems that match the conditions by searching the database. This process uses SQL queries to retrieve information from the database. As output, a list of search results is displayed to the user.

[0472] Step 3:

[0473] The user selects an intelligent system of interest from the search results. As input, the user clicks on a specific system from the system list. As output, the system's detailed information page is displayed. This step includes reviewing details about each system's capabilities, provider information, and available APIs.

[0474] Step 4:

[0475] When a user wants to connect selected intelligent systems, the server configures the necessary communication protocols to establish communication between these systems. The API information of the selected systems is used as input. The server exchanges information using RESTful APIs and configures the necessary communication settings. The output is data exchange between the systems.

[0476] Step 5:

[0477] Users design their workflows using an interactive device. As input, users determine the order and flow of tasks using a visual drag-and-drop interface. As output, the designed workflow is saved and ready for execution by the system. This step provides an intuitive configuration experience through the user interface.

[0478] Step 6:

[0479] Based on the designed workflow, the server automatically executes tasks and provides feedback to the user. The stored workflow data is used as input. The server executes the specified tasks and generates results. The execution results are notified to the user as output. This step involves the execution of automated processes and verification of their results.

[0480] (Application Example 1)

[0481] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0482] In today's complex information society, citizens and users utilize a variety of information processing agents to improve their lives and work in various ways. However, managing and operating these agents is time-consuming, creating a need for a system that allows for centralized and efficient use of agents. Furthermore, there is a need for an environment that facilitates collaboration and interaction between agents, enabling users to create new value.

[0483] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0484] In this invention, the server includes means for setting up an information storage unit for centrally managing multiple information processing agents; means for providing a search engine for searching for and selecting information processing agents within the information storage unit based on a specific request from a user; means for setting up and executing a communication procedure for linking the selected multiple information processing agents; means for providing an external connection unit for the user to define, save, and execute their own work procedures; and means for processing and coordinating external request information to realize interaction between the selected information processing agents. As a result, the user can improve the efficiency of their life and work by effectively managing and coordinating various information processing agents.

[0485] An "information processing agent" is a type of software that is designed to handle specific tasks and automatically performs various processes in response to user requests.

[0486] An "information storage unit" refers to a storage device or database used to centrally store and manage information and data managed within a system.

[0487] The "search engine" is a software function that searches for information processing agents stored in the information storage unit based on requests from the user and provides the optimal result.

[0488] A "communication procedure" is a set of protocols and rules established to facilitate the smooth transmission and reception of data between selected information processing agents.

[0489] The "external connection section" refers to the user interface and connection mechanism for setting, saving, and executing work procedures and workflows that the user has defined independently.

[0490] "External request information" refers to the data and instructions required to enable interaction between selected information processing agents.

[0491] This invention aims to construct a system that centrally manages and efficiently utilizes information processing agents. The server manages multiple information processing agents using a database built in the information storage unit. The server uses a search engine to quickly find the necessary agents in response to user requests. This engine is implemented in Python, and PostgreSQL is used as the database.

[0492] Communication between selected agents is managed by a communication procedure, using a RESTful API as the communication protocol. This enables seamless data exchange between agents. As a result, users can coordinate multiple agents to efficiently manage their work and daily lives.

[0493] The user terminal is equipped with an external connectivity interface built using React Native. Through this interface, users can define, save, and execute their own work procedures. This system ensures that external request information is properly processed and adjusted, providing optimal services tailored to the user's needs.

[0494] As a concrete example, consider traffic management within a smart city. Users can access a traffic management app from their devices and select a traffic information processing agent to understand real-time traffic conditions and receive suggestions for the optimal route. Furthermore, an energy management app can optimize energy consumption.

[0495] By leveraging the generative AI model, users can use the following prompt: "I want to develop a smart city app. Please suggest a combination of AI agents that will allow users to efficiently manage traffic conditions and energy consumption." Using this prompt, the system can suggest the optimal combination of agents and improve the overall usability of the system.

[0496] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0497] Step 1:

[0498] The user accesses the system from a terminal and enters a search query for the information processing agent. This input includes the user's specific requests and objectives. The server receives this query and starts the search engine.

[0499] Step 2:

[0500] The server searches the database in the information storage unit to find the information processing agent that best matches the user's query. It executes the database query using Python and extracts agent information that meets the conditions. As a result of the data processing, a list of agents is output.

[0501] Step 3:

[0502] The user selects the desired agent from a list of agents displayed on the device. The selection process takes into account the agent's function description and provider information.

[0503] Step 4:

[0504] The server configures communication procedures to establish communication between the agents selected by the user. It uses a RESTful API to construct a protocol for data exchange between the selected agents.

[0505] Step 5:

[0506] Users define their own work procedures using the terminal's external connection port and save them to the system. They design workflows based on their needs and save them for future execution.

[0507] Step 6:

[0508] The server processes and coordinates external request information based on stored workflows, enabling interaction between selected information processing agents. Data calculations provide the optimal response.

[0509] Step 7:

[0510] The output presents the user with the final results. For example, the optimal transportation route or information on optimizing energy consumption might be displayed on the user's terminal. This allows the user to take action based on the suggested solutions.

[0511] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0512] This invention provides a system that recognizes user emotions and optimally utilizes artificial intelligence agents accordingly. This system uses an emotion engine to detect the user's emotional state and uses that information to suggest agents and adjust the workflow.

[0513] Specifically, when a user accesses the portal site, they go through a login process and are then taken to the dashboard. On the dashboard, users can use the search bar to find the artificial intelligence agent they need and view descriptions of each agent's functions, provider information, and API information.

[0514] In this system, the portal incorporates an emotion engine, where the server analyzes user input, environmental data, and past usage history to infer the user's current emotional state. Based on this emotional state, the server dynamically recommends the appropriate agent to the user. This recommendation allows the user to choose the agent that best suits their emotions.

[0515] After selecting agents, the user proceeds to build the workflow. The server automatically configures the communication protocol between the selected agents, ensuring seamless sharing of necessary data. The workflow is freely designed by the user through a GUI, and emotional states detected by the emotion engine are also taken into consideration. This allows for the adaptation and adjustment of the flow according to the user's mood.

[0516] For example, when a user is experiencing stress, recommending an agent specializing in relaxation and support can not only improve work efficiency but also contribute to maintaining mental health. Furthermore, for corporate users managing projects, team member assignments and task reallocations can be made in real time, taking into account the emotional state of team members.

[0517] In this way, the present invention provides a portal system for artificial intelligence agents incorporating an emotion engine, enabling flexible and effective use of agents that take into account the user's emotional state.

[0518] The following describes the processing flow.

[0519] Step 1:

[0520] The user logs into the portal site and accesses the dashboard. Here, the user reviews the system overview and looks at the available features.

[0521] Step 2:

[0522] The server checks the user's login information and past usage history. It connects to the emotion engine and collects initial data to infer the user's recent emotional state.

[0523] Step 3:

[0524] The user uses the search bar to search for a specific artificial intelligence agent. The search query includes keywords related to the user's needs.

[0525] Step 4:

[0526] The server processes the user's search query and queries the database to generate a list of suitable artificial intelligence agents. The list also includes a description of each agent's functions and provider information.

[0527] Step 5:

[0528] Simultaneously, the emotion engine analyzes user input information and environmental data acquired from the microphone and camera to determine the user's emotional state. Based on this, it selects the most suitable agent and generates additional recommendation information.

[0529] Step 6:

[0530] The user reviews the list of agents provided by the server and the recommendations from the sentiment engine, and selects the agent that is best suited to them.

[0531] Step 7:

[0532] The server configures the communication protocol to set up coordination between selected agents. It automatically builds API connections based on the selected agents.

[0533] Step 8:

[0534] Users can navigate to a workflow design screen and adjust the flow based on their emotional state. For example, if a user is feeling stressed, the flow can incorporate relaxing tasks or agents that provide enhanced support.

[0535] Step 9:

[0536] The device executes a workflow designed by the user. During execution, the emotion engine monitors the emotional state in real time and dynamically adjusts the flow as needed.

[0537] Step 10:

[0538] The server monitors the progress of tasks and the status of agents, and logs completed tasks and their results. If errors occur, appropriate error handling is performed.

[0539] Step 11:

[0540] After all processes are complete, the user reviews the results and makes final decisions and determines the next actions. Based on these results, the user can improve or save the new workflow.

[0541] (Example 2)

[0542] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0543] In recent years, technologies that utilize multiple intelligent agents to streamline operations have advanced. However, conventional systems do not adequately support the selection of appropriate agents and adjustment of work processes based on the user's emotional state. Therefore, there is a need for systems that can flexibly respond to user requests and feelings.

[0544] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0545] In this invention, the server includes means for setting up an information infrastructure for centrally managing information on multiple intelligent agents, means for providing a search function for searching for and selecting intelligent agents within the information infrastructure based on a specific request from a user, and means for analyzing the user's emotional state and recommending the intelligent agents based on the analysis results. This makes it possible to select an appropriate agent according to the user's emotional state and to adapt and adjust the work process accordingly.

[0546] An "intelligent agent" is a software program that performs specific tasks and responds or performs actions according to the user's requests.

[0547] An "information infrastructure" is a system structure for integrating and managing data from multiple intelligent agents.

[0548] A "user" is a person or organization that uses this system to search for intelligent agents or build workflows.

[0549] The "search function" is a means of searching for and selecting intelligent agents within the information infrastructure based on user requests.

[0550] "Emotional state" refers to information that indicates the user's psychological or emotional state, and is analyzed by the system.

[0551] A "business process" refers to a series of tasks or processes that a user performs using an intelligent agent.

[0552] An "information exchange method" is a protocol or procedure for enabling data communication and collaborative work between multiple intelligent agents.

[0553] This invention constructs a system that utilizes intelligent agents to provide the optimal approach based on the user's emotional state. A server functions as the core of this system, building an information infrastructure that manages information about the intelligent agents. This infrastructure stores the characteristics and functions of each agent. Furthermore, the server uses a search function to select an agent according to the user's specific request. It also uses a generative AI model for emotion analysis to infer the user's emotional state and recommends the most suitable agent based on this information.

[0554] The terminal provides an interface for users to operate this system. On the terminal, users can search for agents and define work processes. Furthermore, the terminal presents the user with agent information obtained from the information infrastructure, assisting in their selection. The selected agents are seamlessly integrated through an information exchange method configured by the server.

[0555] Users can design their own work processes and adjust them as needed through the provided graphical user interface. During this process, advice and task reorganization tailored to their emotional state are provided in real time.

[0556] For example, if a user feels the need to relax, the server will perform emotion analysis and recommend an agent specializing in relaxation. Furthermore, a prompt could be a question posed to the generating AI model such as, "What is the most effective agent action when the user is feeling anxious?"

[0557] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0558] Step 1:

[0559] The server receives login information when a user accesses the portal site. The server accesses the database and verifies the entered user ID and password. In this step, the user ID and password are used as input, and the user's authentication status (authentication successful / failed) is generated as output. If authentication is successful, the dashboard screen is sent to the terminal.

[0560] Step 2:

[0561] The terminal displays a dashboard screen to the user. Here, the user uses the search bar to find the necessary intelligent agent. The keywords entered by the user are used as input data, and a search query is sent from the terminal to the server. The server uses this query to search the agent information infrastructure, generates a list of relevant agents, and outputs it. The terminal then presents this list to the user.

[0562] Step 3:

[0563] The server uses an emotion engine to analyze the user's emotional state. Inputs include the user's mouse and keyboard inputs, as well as environmental data. A generative AI model processes this data to estimate the user's emotional state. The output is the estimated emotional state information. Based on this information, the server dynamically selects the most suitable agent for the user and sends it to the terminal as a recommendation list.

[0564] Step 4:

[0565] The user selects their desired agent from a recommended agent list on their terminal. The selected agent information is sent from the terminal to the server. The server automatically configures the appropriate information exchange method to enable cooperation between the selected agents. In this system, the selected agent information is used as input, and communication protocol configuration data is generated as output.

[0566] Step 5:

[0567] Users design work processes using the terminal's graphical user interface. The server adapts and adjusts the work processes according to the user's emotional state. Using the designed work process data and emotional state data as input, an adjusted work process is generated as output. The work process is optimized in real time and displayed to the user.

[0568] (Application Example 2)

[0569] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0570] In today's information society, there is a demand for flexible and effective use of artificial intelligence that responds to users' emotions and states, but conventional solutions have been insufficient. The challenge lies in appropriately analyzing users' emotions and realizing dynamic recommendations of intelligent functions and optimization of work procedures based on that analysis.

[0571] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0572] In this invention, the server includes means for setting up an information storage device for centrally managing multiple intelligent functions; means for providing a search mechanism for searching for and selecting intelligent functions within the information storage device based on a specific request from a user; means for setting up and executing a communication method for coordinating the selected multiple intelligent functions; and means for providing an emotion analysis mechanism for analyzing the user's emotional state and dynamically recommending intelligent functions accordingly. This enables the flexible and effective use of artificial intelligence in accordance with the user's emotions.

[0573] "Intelligent function" refers to artificial intelligence systems or programs designed to perform specific tasks.

[0574] An "information storage device" refers to a system for efficiently storing and managing data.

[0575] A "search mechanism" refers to a means of searching for and selecting necessary information and functions.

[0576] "Communication method" refers to the protocols and procedures that enable multiple functions or systems to cooperate.

[0577] The "interactive interface" refers to the interface that users use to give instructions to the system or to retrieve information.

[0578] "Emotional state" refers to the emotional state or mental condition that an individual is experiencing.

[0579] An "emotion analysis mechanism" refers to a system or algorithm for analyzing and understanding a user's emotional state.

[0580] "Dynamic recommendation" refers to presenting appropriate options in real time based on the user's state and circumstances.

[0581] The system implementing the present invention consists of a series of programs for analyzing the user's emotional state and dynamically providing intelligent functions based on that analysis.

[0582] The server centrally manages multiple intelligent functions using an information storage device. This makes the intelligent functions of the entire system easily accessible. Based on a user's specific request, the server uses a search mechanism to search for and select the appropriate intelligent function within the information storage device. In this process, the most effective intelligent function is selected.

[0583] The emotion analysis mechanism processes the voice and image data input by the user. Specifically, it uses software such as Python, OpenCV, and TensorFlow to perform real-time emotion analysis. Based on this data analysis, the server aggregates the emotional state and dynamically recommends appropriate intelligence functions. For example, if it determines that the user is feeling stressed, it selects an intelligence function specifically for relaxation.

[0584] The device interacts with the user through a user interface. It assists the user in selecting emotionally appropriate intelligent functions and defining their own work procedures, and effectively coordinates these intelligent functions through communication methods.

[0585] For example, when a user inputs information through a home robot, they can give instructions such as, "Analyze the user's emotions and suggest a word of encouragement." Based on this prompt, the system will begin the analysis and provide appropriate intelligent functions.

[0586] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0587] Step 1:

[0588] The server receives input data from the user. At this stage, input, as well as voice and image data, are acquired through the terminal. Then, the server aggregates the necessary data, taking into account the user's profile, history, etc.

[0589] Step 2:

[0590] The server passes the aggregated input data to the emotion analysis mechanism. This mechanism uses Python, OpenCV, and TensorFlow to analyze the user's emotional state. Specifically, it extracts characteristic patterns from audio data and facial image data, and generates emotion labels (e.g., stress, joy, etc.) from these. The analysis results are used for subsequent processing.

[0591] Step 3:

[0592] Based on the analysis results, the server selects the appropriate intelligent function from within the information storage device. At this stage, a search mechanism searches for the intelligent function that best matches the analyzed emotion label. Then, it collects information about that intelligent function and prepares to set up the communication method.

[0593] Step 4:

[0594] The server provides the terminal with detailed information about the selected intelligent functions. This includes descriptions of the available functions and instructions on how to configure them. Based on this information, the user makes a final decision on which intelligent functions to use through the interactive interface on the terminal.

[0595] Step 5:

[0596] The terminal completes preparations to execute the specified intelligent function based on the user's selection. It sets the communication method and sends commands to ensure seamless coordination of the intelligent functions. Then, once ready, it actually executes the intelligent function and provides the results to the user.

[0597] Step 6:

[0598] After receiving the results of the performed intelligent function, the user inputs feedback into the terminal. This feedback is sent to the server as data for future processing improvements and further sentiment analysis, and is used to train the system.

[0599] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0600] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0601] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0602] [Fourth Embodiment]

[0603] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0604] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0605] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0606] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0607] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0608] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0609] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0610] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0611] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0612] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0613] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0614] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0615] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0616] This invention provides a portal system that allows users to efficiently utilize multiple artificial intelligence agents. Specific embodiments thereof are described below.

[0617] First, users can access the portal site and register, search for, select, and link agents. To automate tasks or improve personal life, users first search for an AI agent that suits their specific needs.

[0618] The server receives search queries from users, analyzes agent information in the database, and presents the user with a list of agents that best match the criteria. Detailed information such as each agent's capabilities, pricing, provider information, and available APIs is also provided. This allows users to quickly identify the agent best suited to their needs.

[0619] Next, the necessary processes can be built based on the agent selected by the user. If the user wants to coordinate multiple agents, the server automatically configures the communication protocol between the selected agents, and the APIs between the agents are seamlessly integrated.

[0620] Furthermore, users can design their own workflows through the GUI and save them. This allows users to efficiently automate their daily tasks according to a schedule. For example, by linking a schedule management agent with an email sending agent, users can create a system that automatically sends reminders via email when a meeting is approaching.

[0621] For users who are employees of companies working from home and are managing project progress, linking a progress management agent with a communication agent that facilitates communication among team members makes it possible to achieve efficient project monitoring and information sharing within the team.

[0622] In this way, the portal system according to the present invention provides an environment in which users can easily combine various artificial intelligence agents based on their own requirements and efficiently make the most of their functions.

[0623] The following describes the processing flow.

[0624] Step 1:

[0625] The user accesses the portal site and enters their authentication information on the login screen. Once the authentication information is entered, the system verifies the user's identity and redirects them to the dashboard.

[0626] Step 2:

[0627] The user enters keywords into the search bar to find an artificial intelligence agent that meets their specific needs.

[0628] Step 3:

[0629] The server receives the search query entered by the user and executes the query in the database. It generates a list of relevant artificial intelligence agents and presents that list to the user.

[0630] Step 4:

[0631] The user selects an agent of interest from the presented agent list and views detailed information. They then review the agent's functions and terms of use and select the agent they wish to use.

[0632] Step 5:

[0633] The user selects multiple agents they wish to use simultaneously and accesses the integration configuration screen.

[0634] Step 6:

[0635] The server automatically generates a communication protocol between the selected agents and configures the connection between APIs based on it. This configuration enables data exchange between agents.

[0636] Step 7:

[0637] Users design their own workflows using the provided GUI. The workflow is formed by specifying the order and trigger conditions for each agent.

[0638] Step 8:

[0639] The terminal executes a saved workflow, sequentially carrying out agent tasks according to user instructions. During task progress, necessary data is shared between agents in real time.

[0640] Step 9:

[0641] The server monitors the task's progress and logs completed steps and errors. For errors, the server takes action such as retrying or suggesting alternative flows.

[0642] Step 10:

[0643] After all tasks are completed, the user reviews the results and decides on the next action. The system's value is maximized by improving workflows or designing new ones based on these results.

[0644] (Example 1)

[0645] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0646] In modern society, the use of intelligent systems plays a crucial role in improving work efficiency and enhancing personal lives. However, technical knowledge and compatibility issues between systems pose obstacles to effectively utilizing and coordinating multiple intelligent systems. Furthermore, accurately understanding the information provided by each intelligent system and finding the optimal combination to suit one's needs is not easy. As a result, users often spend a great deal of time and effort, failing to enjoy the full potential of the systems.

[0647] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0648] In this invention, the server includes means for setting up an information storage device for integrating and managing multiple intelligent systems, means for providing a search device for searching for and selecting intelligent systems within the information storage device based on a specific request from a user, and means for setting up and executing communication control means for coordinating the selected multiple intelligent systems. This makes it possible for users to easily combine and use intelligent systems, construct an optimal processing flow, and realize automation according to their own purposes.

[0649] An "intelligent system" is a technology that has the ability to autonomously process information and propose or implement solutions according to a specific purpose.

[0650] An "information storage device" is a device that centrally manages various types of information related to intelligent systems and allows for quick access to that information as needed.

[0651] A "search device" is a device that provides the function of searching for intelligent systems within an information storage device based on user instructions and selecting an option suitable for the purpose.

[0652] "Communication control means" refers to technology for managing the transmission and reception of data between multiple intelligent systems and for realizing their coordinated operation.

[0653] A "dialogue device" is a device that provides an interface for users to interact with and operate intelligent systems, enabling smooth operation.

[0654] An "operation screen" is a visual user interface that allows users to directly give instructions to the system or input data.

[0655] This system utilizes multiple intelligent systems to support the streamlining of work processes and the automation of personal tasks.

[0656] Users first access the portal site using their device's web browser. This portal site provides an interface for centralized management and collaborative design of intelligent systems.

[0657] The server receives access to the portal site and maintains an information storage device containing information on multiple intelligent systems. Here, the server uses database technology to store information on the functions, provider information, and communication interface information of the intelligent systems. This data is accessed immediately upon user request.

[0658] Users utilize the portal site's search tool to find intelligent systems that meet their needs. By entering keywords and criteria, they can narrow down the search to systems with specific functions and characteristics.

[0659] Based on the search results, the server configures communication control mechanisms to facilitate communication between the specified intelligent systems and integrates the APIs of each system. This process utilizes RESTful APIs and JSON-formatted data exchange.

[0660] Furthermore, users can build their own workflows using a visual interface via an interactive device. For example, by linking a schedule management intelligence system with an email notification intelligence system, it is possible to create a function that automatically sends meeting reminders via email.

[0661] As a concrete example, a possible prompt might be, "I want to create a flow that automatically sends weekly meeting reminder emails using a schedule management agent and an email sending agent." Based on this prompt, the server automatically searches for and configures the relevant intelligent systems to achieve coordination.

[0662] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0663] Step 1:

[0664] The user accesses the portal site using the terminal's web browser. As input, the user enters the portal site's URL into the browser. As output, the portal site's homepage is displayed in the browser. This step provides the user with an entry point to access information about available intelligent systems.

[0665] Step 2:

[0666] Users search for intelligent systems by entering specific keywords or conditions into the search bar on the portal site. As input, users provide search queries based on their needs. The server receives these queries and filters the intelligent systems that match the conditions by searching the database. This process uses SQL queries to retrieve information from the database. As output, a list of search results is displayed to the user.

[0667] Step 3:

[0668] The user selects an intelligent system of interest from the search results. As input, the user clicks on a specific system from the system list. As output, the system's detailed information page is displayed. This step includes reviewing details about each system's capabilities, provider information, and available APIs.

[0669] Step 4:

[0670] When a user wants to connect selected intelligent systems, the server configures the necessary communication protocols to establish communication between these systems. The API information of the selected systems is used as input. The server exchanges information using RESTful APIs and configures the necessary communication settings. The output is data exchange between the systems.

[0671] Step 5:

[0672] Users design their workflows using an interactive device. As input, users determine the order and flow of tasks using a visual drag-and-drop interface. As output, the designed workflow is saved and ready for execution by the system. This step provides an intuitive configuration experience through the user interface.

[0673] Step 6:

[0674] Based on the designed workflow, the server automatically executes tasks and provides feedback to the user. The stored workflow data is used as input. The server executes the specified tasks and generates results. The execution results are notified to the user as output. This step involves the execution of automated processes and verification of their results.

[0675] (Application Example 1)

[0676] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0677] In today's complex information society, citizens and users utilize a variety of information processing agents to improve their lives and work in various ways. However, managing and operating these agents is time-consuming, creating a need for a system that allows for centralized and efficient use of agents. Furthermore, there is a need for an environment that facilitates collaboration and interaction between agents, enabling users to create new value.

[0678] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0679] In this invention, the server includes means for setting up an information storage unit for centrally managing multiple information processing agents; means for providing a search engine for searching for and selecting information processing agents within the information storage unit based on a specific request from a user; means for setting up and executing a communication procedure for linking the selected multiple information processing agents; means for providing an external connection unit for the user to define, save, and execute their own work procedures; and means for processing and coordinating external request information to realize interaction between the selected information processing agents. As a result, the user can improve the efficiency of their life and work by effectively managing and coordinating various information processing agents.

[0680] An "information processing agent" is a type of software that is designed to handle specific tasks and automatically performs various processes in response to user requests.

[0681] An "information storage unit" refers to a storage device or database used to centrally store and manage information and data managed within a system.

[0682] The "search engine" is a software function that searches for information processing agents stored in the information storage unit based on requests from the user and provides the optimal result.

[0683] A "communication procedure" is a set of protocols and rules established to facilitate the smooth transmission and reception of data between selected information processing agents.

[0684] The "external connection section" refers to the user interface and connection mechanism for setting, saving, and executing work procedures and workflows that the user has defined independently.

[0685] "External request information" refers to the data and instructions required to enable interaction between selected information processing agents.

[0686] This invention aims to construct a system that centrally manages and efficiently utilizes information processing agents. The server manages multiple information processing agents using a database built in the information storage unit. The server uses a search engine to quickly find the necessary agents in response to user requests. This engine is implemented in Python, and PostgreSQL is used as the database.

[0687] Communication between selected agents is managed by a communication procedure, using a RESTful API as the communication protocol. This enables seamless data exchange between agents. As a result, users can coordinate multiple agents to efficiently manage their work and daily lives.

[0688] The user terminal is equipped with an external connectivity interface built using React Native. Through this interface, users can define, save, and execute their own work procedures. This system ensures that external request information is properly processed and adjusted, providing optimal services tailored to the user's needs.

[0689] As a concrete example, consider traffic management within a smart city. Users can access a traffic management app from their devices and select a traffic information processing agent to understand real-time traffic conditions and receive suggestions for the optimal route. Furthermore, an energy management app can optimize energy consumption.

[0690] By leveraging the generative AI model, users can use the following prompt: "I want to develop a smart city app. Please suggest a combination of AI agents that will allow users to efficiently manage traffic conditions and energy consumption." Using this prompt, the system can suggest the optimal combination of agents and improve the overall usability of the system.

[0691] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0692] Step 1:

[0693] The user accesses the system from a terminal and enters a search query for the information processing agent. This input includes the user's specific requests and objectives. The server receives this query and starts the search engine.

[0694] Step 2:

[0695] The server searches the database in the information storage unit to find the information processing agent that best matches the user's query. It executes the database query using Python and extracts agent information that meets the conditions. As a result of the data processing, a list of agents is output.

[0696] Step 3:

[0697] The user selects the desired agent from a list of agents displayed on the device. The selection process takes into account the agent's function description and provider information.

[0698] Step 4:

[0699] The server configures communication procedures to establish communication between the agents selected by the user. It uses a RESTful API to construct a protocol for data exchange between the selected agents.

[0700] Step 5:

[0701] Users define their own work procedures using the terminal's external connection port and save them to the system. They design workflows based on their needs and save them for future execution.

[0702] Step 6:

[0703] The server processes and coordinates external request information based on stored workflows, enabling interaction between selected information processing agents. Data calculations provide the optimal response.

[0704] Step 7:

[0705] The output presents the user with the final results. For example, the optimal transportation route or information on optimizing energy consumption might be displayed on the user's terminal. This allows the user to take action based on the suggested solutions.

[0706] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0707] This invention provides a system that recognizes user emotions and optimally utilizes artificial intelligence agents accordingly. This system uses an emotion engine to detect the user's emotional state and uses that information to suggest agents and adjust the workflow.

[0708] Specifically, when a user accesses the portal site, they go through a login process and are then taken to the dashboard. On the dashboard, users can use the search bar to find the artificial intelligence agent they need and view descriptions of each agent's functions, provider information, and API information.

[0709] In this system, the portal incorporates an emotion engine, where the server analyzes user input, environmental data, and past usage history to infer the user's current emotional state. Based on this emotional state, the server dynamically recommends the appropriate agent to the user. This recommendation allows the user to choose the agent that best suits their emotions.

[0710] After selecting agents, the user proceeds to build the workflow. The server automatically configures the communication protocol between the selected agents, ensuring seamless sharing of necessary data. The workflow is freely designed by the user through a GUI, and emotional states detected by the emotion engine are also taken into consideration. This allows for the adaptation and adjustment of the flow according to the user's mood.

[0711] For example, when a user is experiencing stress, recommending an agent specializing in relaxation and support can not only improve work efficiency but also contribute to maintaining mental health. Furthermore, for corporate users managing projects, team member assignments and task reallocations can be made in real time, taking into account the emotional state of team members.

[0712] In this way, the present invention provides a portal system for artificial intelligence agents incorporating an emotion engine, enabling flexible and effective use of agents that take into account the user's emotional state.

[0713] The following describes the processing flow.

[0714] Step 1:

[0715] The user logs into the portal site and accesses the dashboard. Here, the user reviews the system overview and looks at the available features.

[0716] Step 2:

[0717] The server checks the user's login information and past usage history. It connects to the emotion engine and collects initial data to infer the user's recent emotional state.

[0718] Step 3:

[0719] The user uses the search bar to search for a specific artificial intelligence agent. The search query includes keywords related to the user's needs.

[0720] Step 4:

[0721] The server processes the user's search query and queries the database to generate a list of suitable artificial intelligence agents. The list also includes a description of each agent's functions and provider information.

[0722] Step 5:

[0723] Simultaneously, the emotion engine analyzes user input information and environmental data acquired from the microphone and camera to determine the user's emotional state. Based on this, it selects the most suitable agent and generates additional recommendation information.

[0724] Step 6:

[0725] The user reviews the list of agents provided by the server and the recommendations from the sentiment engine, and selects the agent that is best suited to them.

[0726] Step 7:

[0727] The server configures the communication protocol to set up coordination between selected agents. It automatically builds API connections based on the selected agents.

[0728] Step 8:

[0729] Users can navigate to a workflow design screen and adjust the flow based on their emotional state. For example, if a user is feeling stressed, the flow can incorporate relaxing tasks or agents that provide enhanced support.

[0730] Step 9:

[0731] The device executes a workflow designed by the user. During execution, the emotion engine monitors the emotional state in real time and dynamically adjusts the flow as needed.

[0732] Step 10:

[0733] The server monitors the progress of tasks and the status of agents, and logs completed tasks and their results. If errors occur, appropriate error handling is performed.

[0734] Step 11:

[0735] After all processes are complete, the user reviews the results and makes final decisions and determines the next actions. Based on these results, the user can improve or save the new workflow.

[0736] (Example 2)

[0737] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0738] In recent years, technologies that utilize multiple intelligent agents to streamline operations have advanced. However, conventional systems do not adequately support the selection of appropriate agents and adjustment of work processes based on the user's emotional state. Therefore, there is a need for systems that can flexibly respond to user requests and feelings.

[0739] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0740] In this invention, the server includes means for setting up an information infrastructure for centrally managing information on multiple intelligent agents, means for providing a search function for searching for and selecting intelligent agents within the information infrastructure based on a specific request from a user, and means for analyzing the user's emotional state and recommending the intelligent agents based on the analysis results. This makes it possible to select an appropriate agent according to the user's emotional state and to adapt and adjust the work process accordingly.

[0741] An "intelligent agent" is a software program that performs specific tasks and responds or performs actions according to the user's requests.

[0742] An "information infrastructure" is a system structure for integrating and managing data from multiple intelligent agents.

[0743] A "user" is a person or organization that uses this system to search for intelligent agents or build workflows.

[0744] The "search function" is a means of searching for and selecting intelligent agents within the information infrastructure based on user requests.

[0745] "Emotional state" refers to information that indicates the user's psychological or emotional state, and is analyzed by the system.

[0746] A "business process" refers to a series of tasks or processes that a user performs using an intelligent agent.

[0747] An "information exchange method" is a protocol or procedure for enabling data communication and collaborative work between multiple intelligent agents.

[0748] This invention constructs a system that utilizes intelligent agents to provide the optimal approach based on the user's emotional state. A server functions as the core of this system, building an information infrastructure that manages information about the intelligent agents. This infrastructure stores the characteristics and functions of each agent. Furthermore, the server uses a search function to select an agent according to the user's specific request. It also uses a generative AI model for emotion analysis to infer the user's emotional state and recommends the most suitable agent based on this information.

[0749] The terminal provides an interface for users to operate this system. On the terminal, users can search for agents and define work processes. Furthermore, the terminal presents the user with agent information obtained from the information infrastructure, assisting in their selection. The selected agents are seamlessly integrated through an information exchange method configured by the server.

[0750] Users can design their own work processes and adjust them as needed through the provided graphical user interface. During this process, advice and task reorganization tailored to their emotional state are provided in real time.

[0751] For example, if a user feels the need to relax, the server will perform emotion analysis and recommend an agent specializing in relaxation. Furthermore, a prompt could be a question posed to the generating AI model such as, "What is the most effective agent action when the user is feeling anxious?"

[0752] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0753] Step 1:

[0754] The server receives login information when a user accesses the portal site. The server accesses the database and verifies the entered user ID and password. In this step, the user ID and password are used as input, and the user's authentication status (authentication successful / failed) is generated as output. If authentication is successful, the dashboard screen is sent to the terminal.

[0755] Step 2:

[0756] The terminal displays a dashboard screen to the user. Here, the user uses the search bar to find the necessary intelligent agent. The keywords entered by the user are used as input data, and a search query is sent from the terminal to the server. The server uses this query to search the agent information infrastructure, generates a list of relevant agents, and outputs it. The terminal then presents this list to the user.

[0757] Step 3:

[0758] The server uses an emotion engine to analyze the user's emotional state. Inputs include the user's mouse and keyboard inputs, as well as environmental data. A generative AI model processes this data to estimate the user's emotional state. The output is the estimated emotional state information. Based on this information, the server dynamically selects the most suitable agent for the user and sends it to the terminal as a recommendation list.

[0759] Step 4:

[0760] The user selects their desired agent from a recommended agent list on their terminal. The selected agent information is sent from the terminal to the server. The server automatically configures the appropriate information exchange method to enable cooperation between the selected agents. In this system, the selected agent information is used as input, and communication protocol configuration data is generated as output.

[0761] Step 5:

[0762] Users design work processes using the terminal's graphical user interface. The server adapts and adjusts the work processes according to the user's emotional state. Using the designed work process data and emotional state data as input, an adjusted work process is generated as output. The work process is optimized in real time and displayed to the user.

[0763] (Application Example 2)

[0764] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0765] In today's information society, there is a demand for flexible and effective use of artificial intelligence that responds to users' emotions and states, but conventional solutions have been insufficient. The challenge lies in appropriately analyzing users' emotions and realizing dynamic recommendations of intelligent functions and optimization of work procedures based on that analysis.

[0766] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0767] In this invention, the server includes means for setting up an information storage device for centrally managing multiple intelligent functions; means for providing a search mechanism for searching for and selecting intelligent functions within the information storage device based on a specific request from a user; means for setting up and executing a communication method for coordinating the selected multiple intelligent functions; and means for providing an emotion analysis mechanism for analyzing the user's emotional state and dynamically recommending intelligent functions accordingly. This enables the flexible and effective use of artificial intelligence in accordance with the user's emotions.

[0768] "Intelligent function" refers to artificial intelligence systems or programs designed to perform specific tasks.

[0769] An "information storage device" refers to a system for efficiently storing and managing data.

[0770] A "search mechanism" refers to a means of searching for and selecting necessary information and functions.

[0771] "Communication method" refers to the protocols and procedures that enable multiple functions or systems to cooperate.

[0772] The "interactive interface" refers to the interface that users use to give instructions to the system or to retrieve information.

[0773] "Emotional state" refers to the emotional state or mental condition that an individual is experiencing.

[0774] An "emotion analysis mechanism" refers to a system or algorithm for analyzing and understanding a user's emotional state.

[0775] "Dynamic recommendation" refers to presenting appropriate options in real time based on the user's state and circumstances.

[0776] The system implementing the present invention consists of a series of programs for analyzing the user's emotional state and dynamically providing intelligent functions based on that analysis.

[0777] The server centrally manages multiple intelligent functions using an information storage device. This makes the intelligent functions of the entire system easily accessible. Based on a user's specific request, the server uses a search mechanism to search for and select the appropriate intelligent function within the information storage device. In this process, the most effective intelligent function is selected.

[0778] The emotion analysis mechanism processes the voice and image data input by the user. Specifically, it uses software such as Python, OpenCV, and TensorFlow to perform real-time emotion analysis. Based on this data analysis, the server aggregates the emotional state and dynamically recommends appropriate intelligence functions. For example, if it determines that the user is feeling stressed, it selects an intelligence function specifically for relaxation.

[0779] The device interacts with the user through a user interface. It assists the user in selecting emotionally appropriate intelligent functions and defining their own work procedures, and effectively coordinates these intelligent functions through communication methods.

[0780] For example, when a user inputs information through a home robot, they can give instructions such as, "Analyze the user's emotions and suggest a word of encouragement." Based on this prompt, the system will begin the analysis and provide appropriate intelligent functions.

[0781] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0782] Step 1:

[0783] The server receives input data from the user. At this stage, input, as well as voice and image data, are acquired through the terminal. Then, the server aggregates the necessary data, taking into account the user's profile, history, etc.

[0784] Step 2:

[0785] The server passes the aggregated input data to the emotion analysis mechanism. This mechanism uses Python, OpenCV, and TensorFlow to analyze the user's emotional state. Specifically, it extracts characteristic patterns from audio data and facial image data, and generates emotion labels (e.g., stress, joy, etc.) from these. The analysis results are used for subsequent processing.

[0786] Step 3:

[0787] Based on the analysis results, the server selects the appropriate intelligent function from within the information storage device. At this stage, a search mechanism searches for the intelligent function that best matches the analyzed emotion label. Then, it collects information about that intelligent function and prepares to set up the communication method.

[0788] Step 4:

[0789] The server provides the terminal with detailed information about the selected intelligent functions. This includes descriptions of the available functions and instructions on how to configure them. Based on this information, the user makes a final decision on which intelligent functions to use through the interactive interface on the terminal.

[0790] Step 5:

[0791] The terminal completes preparations to execute the specified intelligent function based on the user's selection. It sets the communication method and sends commands to ensure seamless coordination of the intelligent functions. Then, once ready, it actually executes the intelligent function and provides the results to the user.

[0792] Step 6:

[0793] After receiving the results of the performed intelligent function, the user inputs feedback into the terminal. This feedback is sent to the server as data for future processing improvements and further sentiment analysis, and is used to train the system.

[0794] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0795] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0796] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0797] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0798] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0799] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0800] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0801] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0802] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0803] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0804] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0805] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and distributed processing for the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0806] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0807] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0808] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0809] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0810] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0811] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0812] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0813] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0814] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0815] The following is further disclosed regarding the embodiments described above.

[0816] (Claim 1)

[0817] A means of setting up a database for centrally managing information from multiple artificial intelligence agents,

[0818] A means for providing a search engine for searching and selecting artificial intelligence agents in the database based on the specific needs of the user,

[0819] A means of setting up and executing a communication protocol for coordinating multiple selected artificial intelligence agents,

[0820] A means of providing an interface for users to define their own workflows, save them, and execute them,

[0821] A system that includes this.

[0822] (Claim 2)

[0823] The system according to claim 1, further comprising means for providing a user with a functional description of a registered artificial intelligence agent, provider information, and API information.

[0824] (Claim 3)

[0825] The system according to claim 1, further comprising means for a user to select an artificial intelligence agent based on search results and for configuring a collaboration based on the results.

[0826] "Example 1"

[0827] (Claim 1)

[0828] Means for setting up an information storage device for integrated management of multiple intelligent systems,

[0829] A means for providing a search device for searching for and selecting an intelligent system within the information storage device based on the user's specific requests,

[0830] A means for setting and executing communication control means for coordinating multiple selected intelligent systems,

[0831] A means of providing an interactive device for users to construct their own processing flow, record it, and execute it,

[0832] A means for presenting the user with a description of the operation of each intelligent system, provider information, and communication interface information,

[0833] A means by which the user selects an intelligent system based on the search results and configures a collaboration based on those results,

[0834] A system that includes this.

[0835] (Claim 2)

[0836] The system according to claim 1, further comprising means for providing an operation screen for a user to visually design the processing flow.

[0837] (Claim 3)

[0838] The system according to claim 1, further comprising means for setting up automatic communication between intelligent systems.

[0839] "Application Example 1"

[0840] (Claim 1)

[0841] A means for setting up an information storage unit for centrally managing multiple information processing agents,

[0842] Means for providing a search engine for searching for and selecting an information processing agent within the information storage unit based on a specific request from the user,

[0843] A means for setting and executing a communication procedure to coordinate multiple selected information processing agents,

[0844] A means for providing an external connection unit for users to define, save, and execute their own work procedures,

[0845] Means for processing and coordinating external request information in order to realize interaction between selected information processing agents,

[0846] A system that includes this.

[0847] (Claim 2)

[0848] The system according to claim 1, further comprising means for providing users with the functional details, provider information, and connection information of registered information processing agents.

[0849] (Claim 3)

[0850] The system according to claim 1, further comprising means for a user to select an information processing agent based on the search results and for configuring the linkage based on the results.

[0851] "Example 2 of combining an emotion engine"

[0852] (Claim 1)

[0853] A means for setting up an information infrastructure for centrally managing information from multiple intelligent agents,

[0854] Means for providing a search function to search for and select intelligent agents within the information infrastructure based on specific requests from users,

[0855] A means for setting up and executing an information exchange method for coordinating multiple selected intelligent agents,

[0856] A means of providing a user interface for users to define their own business processes, store them, and execute them,

[0857] A means for analyzing the user's emotional state and recommending the intelligent agent based on the analysis results,

[0858] Means for adapting and adjusting the aforementioned work process according to the aforementioned emotional state,

[0859] A system that includes this.

[0860] (Claim 2)

[0861] The system according to claim 1, further comprising means for providing users with a functional description of a registered intelligent agent, provider information, and application program interface information.

[0862] (Claim 3)

[0863] The system according to claim 1, further comprising means for a user to select an intelligent agent based on the search results and for configuring cooperation based on those results.

[0864] "Application example 2 of combining emotional engines"

[0865] (Claim 1)

[0866] A means for setting up an information storage device for centrally managing multiple intelligent functions,

[0867] Means for providing a search mechanism for searching for and selecting intelligent functions within the information storage device based on a specific request from the user,

[0868] A means for setting and executing a communication method to coordinate multiple selected intelligent functions,

[0869] A means of providing an interactive interface for users to define, save, and execute their own work procedures,

[0870] A means to provide an emotion analysis mechanism for analyzing the emotional state of a user and dynamically recommending corresponding intelligence functions,

[0871] A system that includes this.

[0872] (Claim 2)

[0873] The system according to claim 1, further comprising means for providing users with a description of the registered intelligent functions, provider information, and program operation information.

[0874] (Claim 3)

[0875] The system according to claim 1, further comprising means for the user to select an intelligent function based on the search results and for configuring a collaboration based on those results. [Explanation of Symbols]

[0876] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for setting up an information storage unit for centrally managing multiple information processing agents, Means for providing a search engine for searching for and selecting an information processing agent within the information storage unit based on a specific request from the user, A means for setting and executing a communication procedure to coordinate multiple selected information processing agents, A means for providing an external connection unit for users to define, save, and execute their own work procedures, Means for processing and coordinating external request information in order to realize interaction between selected information processing agents, A system that includes this.

2. The system according to claim 1, further comprising means for providing users with the functional details, provider information, and connection information of registered information processing agents.

3. The system according to claim 1, further comprising means for a user to select an information processing agent based on the search results and for configuring the linkage based on the results.

Images