Information processing system, information processing method, and program

Abstract

We provide information processing systems, information processing methods, and programs. [Solution] The information processing system, which includes a computer operated by a user, a server, and a generating AI, comprises: a storage unit that stores skill management information including skill information for multiple skills that constitute a workflow linking multiple types of services; an operation-related information acquisition unit that acquires operation-related information related to the user's operations on the computer operated by the user; a query generation unit that generates queries based on the operation-related information and skill management information; a skill candidate identification unit that inputs the queries to the generating AI and identifies one or more skills output from the generating AI as candidates; and a presentation unit that presents the one or more skills identified by the skill candidate identification unit to the user.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] Patent Document 1 describes an intelligent user support function that infers the needs and preferences of a user in operating a software system or application to support the software user. Patent Document 2 describes a workflow management system that can easily reuse past workflow components and present relevant information in a timely manner. Patent Document 3 describes an AI (Artificial Intelligence) that uses computer vision to recognize applications, screens, and UI (User Interface) elements. Patent Document 4 describes a business process generation system for generating business processes used in a network environment by PaaS (Platform as a Service). [Prior Art Documents] [Patent Documents] [Patent Document 1] Japanese Patent Application Laid-Open No. 2001-510599 [Patent Document 2] Japanese Patent Application Laid-Open No. 2007-188145 [Patent Document 3] Japanese Patent Application Laid-Open No. 2023-545253 [Patent Document 4] Japanese Patent Application Laid-Open No. 2012-242954

Summary of the Invention

Means for Solving the Problems

[0003] According to one embodiment of the present invention, an information processing system is provided. The information processing system may include a storage unit that stores skill management information, which includes skill information for a plurality of skills, each constituting a workflow that links multiple types of services. The information processing system may include an operation-related information acquisition unit that acquires operation-related information related to the user's operations on a computer operated by the user. The information processing system may include a query generation unit that generates queries based on the operation-related information and the skill management information. The information processing system may include a skill candidate identification unit that inputs the queries to a generation AI and identifies one or more skills output from the generation AI as candidates. The information processing system may include a presentation unit that presents the one or more skills identified by the skill candidate identification unit to the user.

[0004] In the information processing system, the operation-related information acquisition unit may acquire user input entered by the user as operation-related information, and the query generation unit may generate a query that includes at least one of the applications running on the computer, the dialogue history between the user and the generated AI, and the user's skill usage history, the operation-related information, and the skill management information. In any of the information processing systems, if multiple applications are running on the computer, the query generation unit may generate a query that includes the application information, which includes the multiple applications and the hierarchical relationship between the multiple applications. In any of the information processing systems, the query generation unit may generate a query that includes the dialogue history, which includes the dialogue history with the user from a predetermined period prior to the time the query was generated, and dialogue trend information generated from the dialogue history with the user before the predetermined period prior to the time the query was generated. In any of the information processing systems, the query generation unit may generate a query that includes the usage history, which includes the ranking of skills used by the user in the past.

[0005] In any of the above-mentioned information processing systems, the operation-related information acquisition unit may acquire at least one of the applications running on the computer and the information displayed on the computer's screen as the operation-related information, and the query generation unit may generate the query which includes at least one of the applications running on the computer and the information displayed on the computer's screen, and the skill management information. The query generation unit may further generate the query which includes the user's skill usage history.

[0006] In any of the above information processing systems, the operation-related information acquisition unit may acquire user input entered by the user as operation-related information, the query generation unit may generate a parameter query for generating candidate parameters which are candidates for parameters to be set in the target skill, based on the required parameter item information indicating the parameter items required to execute the target skill and the operation-related information, and the information processing system may include a candidate parameter identification unit that identifies the candidate parameters based on the parameter query, and a setting unit that sets the candidate parameters identified by the candidate parameter identification unit in the target skill. The query generation unit may further generate the parameter query which includes at least one of characteristic information indicating the characteristics of the target skill, the history of interaction with the user, and the history of parameters previously set for the target skill.

[0007] Any of the above information processing systems may further include the generation AI.

[0008] According to one embodiment of the present invention, an information processing method performed by a computer is provided. The information processing method may include a storage step of storing skill management information in a storage unit, which includes skill information for a plurality of skills, each constituting a workflow that links multiple types of services. The information processing method may include an operation-related information acquisition step of acquiring operation-related information related to the user's operations on a computer operated by the user. The information processing method may include a query generation step of generating a query based on the operation-related information and the skill management information. The information processing method may include a selection step of identifying one or more skills as candidates from the plurality of skills based on the query. The information processing method may include a presentation step of presenting the one or more skills identified in the selection step to the user as candidates.

[0009] According to one embodiment of the present invention, a program is provided for causing a computer to execute the information processing method.

[0010] It should be noted that the above summary of the invention does not list all the necessary features of the present invention. Furthermore, subcombinations of these features may also constitute an invention. [Brief explanation of the drawing]

[0011] [Figure 1] An example of information processing system 100 is shown in outline. [Figure 2] This is an explanatory diagram for describing the skill recommendation function (user-active version) of the information processing system 100. [Figure 3] This is an explanatory diagram for describing the skill recommendation function (user-active version) of the information processing system 100. [Figure 4] This is an explanatory diagram for describing the skill recommendation function (user-passive version) of the information processing system 100. [Figure 5] This is an explanatory diagram illustrating the parameter generation function during skill execution by the information processing system 100. [Figure 6] This is an explanatory diagram illustrating the parameter generation function during skill execution by the information processing system 100. [Figure 7] This is an explanatory diagram illustrating the parameter generation function during skill execution by the information processing system 100. [Figure 8] This is an explanatory diagram illustrating an example of input and output to the generated AI 400 by the information processing system 100. [Figure 9] This is an explanatory diagram illustrating an example of input and output to the generated AI 400 by the information processing system 100. [Figure 10] This is an explanatory diagram illustrating an example of input and output to the generated AI 400 by the information processing system 100. [Figure 11] An example of the functional configuration of the information processing system 100 is shown in general terms. [Figure 12] An example of the processing flow by the information processing system 100 is shown in general terms. [Figure 13] An example of the processing flow by the information processing system 100 is shown in general terms. [Figure 14] An example of the processing flow by the information processing system 100 is shown in general terms. [Figure 15] A schematic example of the hardware configuration of a computer 1200 that functions as an information processing system 100 is shown. [Modes for carrying out the invention]

[0012] The present invention will be described below through embodiments, but these embodiments are not intended to limit the scope of the claims. Furthermore, not all combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0013] Conventionally, generative AI services such as ChatGPT (registered trademark) can accept any prompt from a user and present an answer along with the question, but it has been difficult to understand the situation the user is in or the issues they are facing and return a considerate proposal or answer. Also, the user himself / herself has had to describe the prompt and "master" the generative AI. That is, it has been difficult for only users with the ability to master the generative AI (prompt engineering skills) to draw out the power of the generative AI, and the "super" efficiency improvement of business by the generative AI has not been achieved. The information processing system 100 according to the present embodiment can function as a business support platform that, for example, reads business issues from the current situation / past usage history, etc. by the generative AI when the user selects a strip of a part the user is interested in on the SaaS application screen used in the business, and presents an appropriate application or self-made workflow. The strip selection may include a range text selection which is an act of selecting a range of text with an operation feeling similar to when selecting text to copy. The strip selection may include, for example, a range screen shot form selection which is an act of selecting a range by dragging from the upper left pivot point to the lower right pivot point with an operation feeling similar to the function of taking a screen shot by specifying the range assigned to "Windows" + "Shift" + "S" in Windows (registered trademark) or the function of taking a screen shot by specifying the range assigned to "Shift" + "Command" + "4" in Mac (registered trademark).

[0014] FIG. 1 schematically shows an example of the information processing system 100. The information processing system 100 may be realized by a computer 200 operated by a user 202. The information processing system 100 realized by the computer 200 supports the operation of the computer 200 by the user 202.

[0015] In the form where the computer 200 accesses the server 300 to execute various processes, the information processing system 100 may be realized by the server 300. For example, in the case of a thin client environment, most functions are arranged in the server 300, and the computer 200 executes limited functions such as input and display. However, the information processing system 100 realized in the server 300 assists the user 202 in operating the computer 200 for the processes in the server 300. The computer 200 and the server 300 may communicate via the network 20. The network 20 may include the cloud. The network 20 may include the Internet. The network 20 may include a mobile communication network. The network 20 may include a LAN (Local Area Network).

[0016] The information processing system 100 has a plurality of skills available to the user 202. The skills may constitute a workflow that coordinates multiple types of services. The services may be so-called SaaS (Software as a Service). The services may be services provided by an application. Different workflows are constituted for each skill. For example, a certain skill constitutes a workflow that coordinates a web conferencing application that provides a web conferencing service, an AI application that provides a service for transcribing the voice of a web conference to create minutes, an AI application that provides a summarization service for summarizing the minutes, and a CRM application that provides a CRM service for storing and managing the conference and summarized minutes. The workflow may coordinate any number of services. The workflow may coordinate any type of services. The information processing system 100 may generate and manage a plurality of skills using so-called iPaaS (Integration Platform as a Service).

[0017] The information processing system 100 stores skill management information, which may be referred to as skill information, containing information on multiple skills. Skill information may include skill identification information that can identify a skill. Skill identification information may be, for example, the name of the skill. Skill information may include a descriptive text that explains the skill. For example, the skill information for a skill that constitutes a workflow linking a web conferencing application, an AI application that provides a service for transcribing web conference audio to create meeting minutes, an AI application that provides a summarization service for summarizing the meeting minutes, and a CRM application that provides a CRM service for saving and managing meetings and summarized meeting minutes, includes a descriptive text that explains that the skill provides a web conferencing service, a service for transcribing web conference audio to create meeting minutes, a summarization service for summarizing the meeting minutes, and a CRM service for saving and managing meetings and summarized meeting minutes. Skill information may include information on each of the multiple types of services (sometimes referred to as service information) that the workflow constituting the skill links. Service information may include service identification information that can identify a service. Service identification information may be, for example, the name of the service. Service information may include the type of service. Service information may include a descriptive text describing the service. Service information may include information about the application that implements the service. The application information may include application identifier information that identifies the application. The application identifier information may be the name of the application. The application information may include the type of application. The application information may include a descriptive text describing the application.

[0018] Furthermore, skills may correspond to applications. That is, the information processing system 100 may have multiple skills, each constituting a workflow, and multiple skills, each constituting an application, and the skill management information may include skill information for multiple skills, each constituting a workflow, and skill information for multiple skills, each constituting an application.

[0019] For example, the information processing system 100, in response to input from user 202 (sometimes referred to as user input), identifies one or more recommended skills from among several skills and presents them to user 202 as candidates. For example, the information processing system 100 presents the identified skills to user 202 in order of recommendation.

[0020] The information processing system 100, for example, without explicit input from the user 202, identifies one or more recommended skills from among several skills based on the open applications and information displayed on the screen, and presents them to the user 202 as candidates. For example, the information processing system 100 presents the identified skills to the user 202 in the order of recommendation.

[0021] The information processing system 100 may use the generation AI 400 to assist the user 202 in operating the computer 200. The generation AI 400 may be an LLM (Large Language Model). The generation AI 400 may be a so-called interactive generation AI. The generation AI 400 may be a generation AI that combines an LLM (Large Language Model) with an interactive UI. Examples of the generation AI 400 include ChatGPT, Gemini, and Claude®, but it is not limited to these and may be any existing generation AI. Furthermore, the generation AI 400 may be a generation AI created for the information processing system 100. The information processing system 100 may communicate with the generation AI 400 via the network 20. The information processing system 100 may also be equipped with the generation AI 400.

[0022] By appropriately inputting the necessary information to the generating AI 400, highly accurate output can be obtained. The information processing system 100 may understand the situation and challenges that user 202 is facing, generate queries that allow the generating AI 400 to select the skills recommended for that situation and challenges, and input these queries into the generating AI 400.

[0023] For example, the information processing system 100 generates a query that includes instructions to select a recommended skill for the operation-related information from among multiple skills whose skill information is included in the skill management information, based on operation-related information related to the user 202's operations on the computer 200 operated by the user 202 and skill management information of multiple skills possessed by the information processing system 100.

[0024] Operation-related information may be user input. Operation-related information may be information indicating the status of computer 200, such as applications running on computer 200 or information displayed on the screen of computer 200.

[0025] The information processing system 100 generates a query that includes, for example, operation-related information, skill management information, and text indicating an instruction to output one or more recommended skills from among multiple skills whose skill information is included in the skill management information based on the operation-related information, and inputs it to the generating AI 400. The information processing system 100 presents the one or more skills output from the generating AI 400 to the user 202. This makes it possible to present the user 202 with skills as candidates that are suitable for the situation the user 202 is in and the challenges they are facing.

[0026] When an existing generative AI is given input indicating a request for recommended applications to achieve a certain objective, it outputs a response that includes recommended applications. This is because existing generative AIs learn by acquiring information on various applications from the web, etc. On the other hand, existing generative AIs have not been able to select and recommend one or more skills from multiple skills, each constituting a workflow, that are implemented on a specific computer 200. In contrast, for example, the information processing system 100 according to this embodiment has a function to generate and manage multiple skills that constitute a workflow, or a function to manage multiple generated skills, using iPaaS, etc. Furthermore, by having a function to include skill management information in the query input to the generative AI 400, it is possible to have the generative AI 400 select and output recommended skills for operation-related information and present them to the user 202. The variations in workflows will be as numerous as the combination of linked services, and the number can be extremely large compared to the number of applications, etc. In other words, the number of skills can be extremely large. If the user 202 can make full use of this vast number of skills, the convenience of the user will be greatly improved, but the hurdle to mastering them will be extremely high. In contrast, the information processing system 100 can select and present to the user 202 the recommended skills for operation-related information from a vast number of skills, thereby supporting the user 202 in mastering the skills and greatly contributing to improving the user's convenience.

[0027] Figures 2 and 3 are explanatory diagrams illustrating the skill recommendation function (user-active version) of the information processing system 100. The information processing system 100 presents recommended skills in response to explicit user input from user 202.

[0028] In the examples shown in Figures 2 and 3, the email application window 212 and the agent window 214, which is the window for the agent assisting user 202, are displayed. Figure 2 shows the agent window 214 in its default state, where no user input has been made. In the agent window 214's default state, multiple skills may be displayed in a default order.

[0029] Figure 3 shows the state of the agent window 214 when the same content as the email text is entered into the agent window 214. The information processing system 100 generates a query that includes the entered email text, skill management information, and the fact that the application being used is an email application, in response to user input, and inputs it into the generation AI 400, and retrieves one or more skills output from the generation AI 400. In Figure 3, the generation AI 400 outputs the skill "Automatic generation of approval documents" which is recommended for the content of the email text, and the information processing system 100 sorts and presents multiple skills in an order prioritizing "Automatic generation of approval documents". This allows the system to proactively notify user 202 of the skills they should use in response to the email text, thereby improving user convenience.

[0030] Figure 4 is an explanatory diagram illustrating the skill recommendation function (user-passive version) of the information processing system 100. The information processing system 100 identifies recommended skills from the status of the computer 200 and presents them to the user 202, even without explicit input from the user 202.

[0031] In the example shown in Figure 4, the calendar app window 222, which is the window of the calendar app, is displayed, indicating that the appointment "A Company Meeting" registered in the calendar has been selected by user 202. In response to user 202 selecting "A Company Meeting," the information processing system 100 generates a query that includes the content of the selected appointment, skill management information, and the fact that the app being used is the calendar app, inputs it into the generating AI 400, and retrieves one or more skills output from the generating AI 400. In Figure 4, the generating AI 400 outputs the skills "Meeting Minutes Summarization & CRM Saving," "Meeting Slide Creation," and "Schedule Adjustment Contact from Available Slots," which are recommended for the appointment "A Company Meeting," and the information processing system 100 presents the agent window 224 containing these skills. This allows the system to proactively notify user 202 of recommended skills according to user 202's actions, thereby improving user convenience.

[0032] As described above, the information processing system 100 can present user 202 with candidate skills suitable for the situation and challenges the user 202 is facing. User 202 can then select a desired skill from the presented candidate skills and proceed with using that skill. Some skills may start executing upon selection, while others may not start executing until user 202 inputs the minimum required parameters. In such cases, the information processing system 100 may further assist in inputting the minimum required parameters for the skill.

[0033] Figures 5, 6, and 7 are explanatory diagrams illustrating the parameter generation function during skill execution by the information processing system 100. When executing a skill, the information processing system 100 may identify and set the minimum necessary parameters based on user input and the status of the computer 200.

[0034] In the example shown in Figure 5, user 202 inputs "I need to buy some vegetables" into the agent window 230, and the information processing system 100 presents several recommended skills. Figure 6 shows the state where user 202 has selected the skill "Ask AI". The information processing system 100 inputs the user input "I need to buy some vegetables" into the generating AI 400, and displays the output from the generating AI 400 in the agent window 230. In the example shown in Figure 6, the output is "Which vegetables are you planning to buy? Also, please tell us if you have any cooking plans. We can also tell you about recommended ways to use them."

[0035] Figure 7 shows the state where the skill "Add task to task management" has been selected by user 202. Here, we will explain that the minimum parameter items required to execute the skill "Add task to task management" are the task name and details. Based on the user input "I have to buy vegetables," the displayed information "Which vegetables are you planning to buy? Also, please tell us if you have any cooking plans. We can also tell you about recommended ways to use them," and the required parameter items of task name and details, the information processing system 100 generates a parameter query to generate recommended parameters for setting the skill "Add task to task management." For example, the information processing system 100 generates a parameter query that includes user input, displayed information, required parameter item information indicating the required parameter items, and instructions to output recommended parameters for the required parameter items indicated by the required parameter item information based on the user input and displayed information. The information processing system 100 inputs the generated parameter query to the generation AI 400, retrieves the parameters output by the generation AI 400, and sets them to the skill "Add task to task management." In the example shown in Figure 7, the parameter "Task Name" is set to "Purchase Vegetables," and the parameter "Details" is set to "Purchase necessary vegetables at the supermarket." This supports the use of skills by user 202 and contributes to improving user 202's operating environment.

[0036] Figure 8 is an explanatory diagram illustrating an example of input and output to the Generating AI 400 by the information processing system 100. Here, we will explain the input and output of the information processing system 100 to the Generating AI 400 for realizing the skill recommendation function (user-active version).

[0037] The information processing system 100 inputs a query generated by the generating AI 400 to output a recommended skill from among multiple skills whose skill information is included in the skill management information 112 based on the user input 111, and obtains the recommended skill 121 from the generating AI 400. The information processing system 100 may input a query to the generating AI 400 that includes, for example, the user input 111, the skill management information 112, and text indicating the content to output one or more recommended skills from among multiple skills whose skill information is included in the skill management information 112 based on the user input 111. The information processing system 100 may further input a query to the generating AI 400 that includes text indicating the content to output the recommended order if there are multiple recommended skills.

[0038] User input 111 may be an image. For example, user input 111 may be an image generated by taking a screenshot of part or all of the screen of computer 200. User input 111 may be an image of a region of the computer 200 screen specified by user 202. User input 111 may be a strip-selected image. User input 111 may be an image selected by range screenshot format selection. User input 111 may be text. User input 111 may be text selected by range text selection. User input 111 may be audio. User input 111 may include other types of input from user 202.

[0039] The skill management information 112 may include skill information for multiple skills available to the user 202, which is held by the information processing system 100. The information processing system 100 may include the skill management information 112 in the query as the basis for the skills to be selected by the generating AI 400.

[0040] The information processing system 100 may generate a query that further includes application information 113 open on the computer 200. The application information 113 may be information about an application running on the computer 200. The application information may include application identification information that can identify the application. The application information may include application type information that indicates the type of application. Various types of applications are known, including email applications, calendar applications, web conferencing applications, business management applications, etc., but in this embodiment, the application type may include all known types, or it may include only a portion of all known types. It is considered that there is a strong correlation between the applications running on the computer 200 and the interests of the user 202. Therefore, by inputting the application information 113 into the generating AI 400 in addition to the user input 111 and skill management information 112, it is possible to increase the likelihood that the generating AI 400 will select skills that are more suitable to the user 202's interests compared to when no input is provided. In other words, by inputting the application information 113 into the generating AI 400, it is possible to recommend skills that are more suitable for the user 202 at that time.

[0041] If multiple applications are running on computer 200, the information processing system 100 may include application information 113 containing information about multiple applications in its query. In this case, the application information 113 may include the hierarchical relationships of the multiple applications. For example, the application information 113 may include the hierarchical order of the windows of the multiple applications. Applications with shallower hierarchies are more likely to be of interest to user 202 than applications with deeper hierarchies. For example, if a calendar application is at the top level, a business management application is at the level below that, and an email application is at the level below that, user 202 is most likely to be most interested in the calendar, i.e., scheduling, followed by business management, and then email. Therefore, by inputting application information 113 containing the hierarchical relationships of multiple applications into the generating AI 400, it is possible to increase the likelihood that the generating AI 400 will select skills appropriate to user 202's multiple interests and the degree of those interests, compared to not inputting the application information. Furthermore, when multiple applications are running on the computer 200, the information processing system 100 may include application information 113 in the query that contains information only about the top-level application. When multiple applications are running on the computer 200, the information processing system 100 may include application information 113 in the query that contains information only about the active application.

[0042] The application information 113 may include the total usage time, which is the sum of the time the application has been used since its execution began. If multiple applications are running on computer 200, the application information 113 may include the total usage time for each of the multiple applications. Applications with longer total usage times are more likely to be of greater interest to user 202 than applications with shorter total usage times. Therefore, by inputting application information 113 containing the total usage times of multiple applications into the generating AI 400, it is possible to increase the likelihood that the generating AI 400 will select skills that are appropriate to user 202's multiple interests and the degree of those interests, compared to not inputting the information.

[0043] The application information 113 may include the elapsed time since the application started running. If multiple applications are running on computer 200, the application information 113 may include the elapsed time for each of the multiple applications. Applications with shorter elapsed times are more likely to be of greater interest to user 202 than applications with longer elapsed times. Therefore, by inputting application information 113 containing the elapsed times of multiple applications into the generating AI 400, it is possible to increase the likelihood that the generating AI 400 will select skills that are appropriate to user 202's multiple interests and the degree of those interests, compared to not inputting the information.

[0044] The application information 113 may include the window size of the application. If multiple applications are running on computer 200, the application information 113 may include the window size of each of the multiple applications. Applications with larger window sizes are more likely to be of interest to user 202 than applications with smaller window sizes. Therefore, by inputting application information 113 including the window sizes of multiple applications into the generating AI 400, it is possible to increase the likelihood that the generating AI 400 will select skills that are appropriate to user 202's multiple interests and the degree of those interests, compared to not inputting the information.

[0045] The information processing system 100 may generate queries that further include past dialogue history 114. Dialogue history 114 is a history of past dialogues with user 202. The information processing system 100 may store the history of dialogues with user 202 and include dialogue history 114 in queries. Dialogue history 114 may be a history of dialogues between the information processing system 100 and user 202. That is, dialogue history 114 may be a history of input and output between the information processing system 100 and user 202. Dialogue history 114 may be a history of dialogues between user 202 and the generating AI 400 mediated by the information processing system 100. Dialogue history 114 includes information such as what user 202 requested from the information processing system 100, what user 202 felt about the output from the information processing system 100, what user 202 requested from the generating AI 400, and what user 202 felt about the output from the generating AI 400. Therefore, by inputting the dialogue history 114 into the generating AI 400, it is possible to increase the likelihood that the generating AI 400 will select skills that align with the user 202's preferences, or to decrease the likelihood that the generating AI 400 will select skills that contradict the user 202's preferences.

[0046] The information processing system 100 may include in its queries dialogue history 114 from a predetermined period prior to a certain point in time. Since older dialogue history 114 may not match the current user 202's interests, limiting input to only newer dialogue history 114 can increase the likelihood that the generating AI 400 will select skills that are more appropriate to the current user 202's interests.

[0047] The information processing system 100 may include in its queries dialogue history 114 from a predetermined period prior to a certain point in time, and dialogue trend information generated from dialogue history 114 prior to a predetermined period in time. The dialogue trend information may include information summarizing the dialogue history 114. The dialogue trend information may include information on characteristic topics extracted from the dialogue history 114. The dialogue trend information may include topics that appear frequently in the dialogue history 114, extracted from the dialogue history 114. Older dialogue history 114 is less likely to be relevant to the current user 202's interests compared to newer dialogue history 114, but it may still be relevant. Therefore, by inputting the newer dialogue history 114 as is and inputting the older dialogue history 114 as dialogue trend information, the older dialogue history 114 can be appropriately utilized, potentially increasing the likelihood that the generating AI 400 will select skills that are appropriate to the current user 202's interests.

[0048] The information processing system 100 may generate queries that further include the past skill call history 115. The call history 115 is the usage history of skills that user 202 has called and used in the past. The information processing system 100 may store the skill call history 115 by user 202 and may include the call history 115 in queries. The call history 115 contains information on skills that user 202 frequently uses among multiple skills. Therefore, by inputting the call history 115 into the generating AI 400, it is possible to increase the likelihood that the generating AI 400 will select skills that meet user 202's preferences.

[0049] The call history 115 may include information that identifies the skill used by user 202, as well as the timing of the skill's use. The timing of the skill's use may be a date or a date and time. The call history 115 may include the number of times the skill used by user 202 has been used. The call history 115 may include the order in which the skills used by user 202 have been used. The call history 115 may include skills that user 202 has used among multiple skills. In other words, the call history 115 includes skills that user 202 has used, but does not include skills that user 202 has not used. By including these in the call history 115, in addition to the skills used by user 202, user 202's skill usage tendencies can also be input to the generating AI 400, potentially increasing the likelihood that the generating AI 400 will select skills that meet user 202's preferences.

[0050] Figure 9 is an explanatory diagram illustrating an example of input and output to the generating AI 400 by the information processing system 100. Here, we will explain the input and output to the generating AI 400 by the information processing system 100 for realizing the skill recommendation function (user passive version).

[0051] The information processing system 100 inputs a query generated by the generating AI 400 to output a recommended skill from among multiple skills whose skill information is included in the skill management information 112, based on the application information 113 open on the computer 200 and the display information 116 displayed on the screen of the computer 200, and obtains the recommended skill 121 from the generating AI 400. The information processing system 100 may input a query to the generating AI 400 that includes, for example, the application information 113, the display information 116, the skill management information 112, and text indicating that the system should output one or more recommended skills from among multiple skills whose skill information is included in the skill management information 112 based on the application information 113 and the display information 116. The information processing system 100 may further input a query to the generating AI 400 that includes text indicating that the recommended order should also be output if there are multiple recommended skills.

[0052] The information processing system 100 may input the display information 116 to the generation AI 400 only if it has obtained approval from user 202 to input the display information 116 to the generation AI 400. If it has not obtained approval from user 202 to input the display information 116 to the generation AI 400, the information processing system 100 generates a query that does not include the display information 116 and inputs it to the generation AI 400.

[0053] The application information 113 may be information about an application running on the computer 200. In situations where there is no explicit input from the user 202, the application running on the computer 200 can be considered useful information for estimating the user 202's interests. Therefore, by inputting the application information 113 into the generating AI 400, the generating AI 400 can be enabled to select skills that are appropriate to the user 202's interests.

[0054] If multiple applications are running on computer 200, the information processing system 100 may include application information 113 containing information about multiple applications in the query. In this case, the application information 113 may include the hierarchical relationships of the multiple applications. For example, the application information 113 may include the hierarchical order of the windows of the multiple applications. Furthermore, when multiple applications are running on computer 200, the information processing system 100 may include application information 113 containing information only about the top-level application in the query. When multiple applications are running on computer 200, the information processing system 100 may include application information 113 containing information only about the active application in the query.

[0055] The application information 113 may include total usage time, which is the sum of the time the application has been used since the application started running. If multiple applications are running on computer 200, the application information 113 may include the total usage time for each of the multiple applications.

[0056] The application information 113 may include the elapsed time since the application started running. If multiple applications are running on computer 200, the application information 113 may include the elapsed time for each of the multiple applications.

[0057] The application information 113 may include the size of the application's window. If multiple applications are running on computer 200, the application information 113 may include the size of each window for each of the applications.

[0058] The display information 116 may include all the information displayed on the screen of the computer 200. The display information 116 may include some of the information displayed on the screen of the computer 200. The display information 116 may include text from all the information displayed on the screen of the computer 200. The information processing system 100 includes, for example, information from the information displayed on the screen of the computer 200 that is highly likely to be of interest to the user 202 as the display information 116 in the query. For example, the information processing system 100 includes information selected by the user 202 from the information displayed on the screen of the computer 200 in the query. For example, when multiple windows are displayed on the screen of the computer 200, the information processing system 100 may include information contained in the active window in the query, information contained in the top-level window in the query, or information contained in the window where the cursor is located in the query. In situations where there is no explicit input from the user 202, the display information 116 can be said to be useful information for estimating the user 202's interests. Therefore, by obtaining the user 202's approval and inputting the display information 116 into the generating AI 400, the generating AI 400 can select skills that are appropriate to the user 202's interests.

[0059] The skill management information 112 may include skill information for multiple skills available to the user 202, which is held by the information processing system 100. The information processing system 100 may include the skill management information 112 in the query as the basis for the skills to be selected by the generating AI 400.

[0060] The information processing system 100 may generate a query that further includes the skill call history 115. The call history 115 is the usage history of skills that user 202 has called and used in the past. The information processing system 100 may store the skill usage history by user 202 and may include the call history 115 in the query.

[0061] The call history 115 may include information that identifies the skill used by user 202, as well as the timing of the skill's use. The timing of the skill's use may be a date or a date and time. The call history 115 may include the number of times a skill has been used by user 202. The call history 115 may include the order of the number of times a skill has been used by user 202. The call history 115 may include skills that user 202 has used at some point among multiple skills. In other words, the call history 115 includes skills that user 202 has used at some point, but does not include skills that user 202 has not used.

[0062] Furthermore, the information processing system 100 may generate queries that include a dialogue history, which is a history of past interactions with user 202. The information processing system 100 may store a history of interactions with user 202 and include the dialogue history in the queries. The dialogue history may be a history of interactions between the information processing system 100 and user 202. That is, the dialogue history may be a history of inputs and outputs between the information processing system 100 and user 202. The dialogue history may be a history of interactions between user 202 and the generating AI 400, mediated by the information processing system 100. The information processing system 100 may include the dialogue history from a predetermined period of time prior to the current point in the queries. The information processing system 100 may include the dialogue history from a predetermined period of time prior to the current point in the queries, as well as dialogue trend information generated from the dialogue history prior to the predetermined period of time prior to the current point in the queries. The dialogue trend information may include information summarizing the dialogue history. The dialogue trend information may include information on characteristic topics extracted from the dialogue history. Dialogue trend information may include topics that appear frequently in the dialogue history, extracted from the dialogue history.

[0063] Figure 10 is an explanatory diagram illustrating an example of input and output to the generated AI 400 by the information processing system 100. Here, we will explain the input and output to the generated AI 400 by the information processing system 100 in order to realize the parameter generation function.

[0064] The information processing system 100 inputs a query generated by the generating AI 400 to output recommended parameters for the parameter items indicated by the required parameter item information 117 based on the user input 111, and obtains the recommended parameters 122 from the generating AI 400. The information processing system 100 may input a query to the generating AI 400 that includes, for example, the user input 111, the required parameter item information 117, and text indicating the content to output recommended parameters for the parameter items indicated by the required parameter item information 117 based on the user input 111. If there are multiple recommended parameters, the information processing system 100 may further input a query to the generating AI 400 that includes text indicating the content to output the recommended order.

[0065] User input 111 may be an image. For example, user input 111 may be an image generated by taking a screenshot of part or all of the screen of computer 200. User input 111 may be an image of a region of the computer 200 screen specified by user 202. User input 111 may be a strip-selected image. User input 111 may be an image selected by range screenshot format selection. User input 111 may be text. User input 111 may be text selected by range text selection. User input 111 may be audio. User input 111 may include other types of input from user 202.

[0066] The information processing system 100 may generate queries that further include past dialogue history 114. Dialogue history 114 is a history of past dialogues with user 202. The information processing system 100 may store the history of dialogues with user 202 and include dialogue history 114 in queries. Dialogue history 114 may be a history of dialogues between the information processing system 100 and user 202. That is, dialogue history 114 may be a history of input / output between the information processing system 100 and user 202. Dialogue history 114 may be a history of dialogues between user 202 and the generating AI 400 mediated by the information processing system 100. The information processing system 100 may include dialogue history 114 from a predetermined period of time prior to the current point in the query. The information processing system 100 may include dialogue trend information generated from dialogue history 114 from a predetermined period of time prior to the current point in the query, and from dialogue history 114 prior to the predetermined period of time prior. Dialogue trend information may include information summarizing dialogue history 114. Dialogue trend information may include information on characteristic topics extracted from dialogue history 114. Dialogue trend information may include topics with high frequency of appearance from dialogue history 114, extracted from dialogue history 114.

[0067] The information processing system 100 may generate a query that further includes characteristic information 118. The characteristic information 118 indicates the characteristics of the target skill. The characteristic information 118 may indicate the characteristics of multiple services with which the workflow composed of the target skill interacts. The characteristic information 118 may indicate the characteristics of each application of the multiple services. For example, if the target skill constitutes a workflow that interacts with a web conferencing application that provides a web conferencing service, an AI application that provides a service that transcribes the audio of the web conference and creates meeting minutes, an AI application that provides a summarization service that summarizes the meeting minutes, and a CRM application that provides a CRM service that saves and manages the meeting and the summarized meeting minutes, the characteristic information 118 may include web conferencing, transcription, summarization, and saving. By including characteristic information 118 in the query in addition to the required parameter item information 117, the generating AI 400 can more easily generate parameters appropriate to the characteristics for multiple parameter items of the target skill, and the likelihood that the parameters presented to the user 202 will be adopted as is by the user 202 can be increased.

[0068] The information processing system 100 may generate a query that further includes the parameter setting history 119. The parameter setting history 119 is a history of parameters previously set for the parameter items indicated by the required parameter item information 117. The information processing system 100 may accumulate the parameter setting history for each of the multiple skills and include the parameter setting history 119 in the query. The information processing system 100 may include in the query all parameters that have been previously set for the parameter items indicated by the required parameter item information 117. The information processing system 100 may include in the query the parameter setting history 119 from a predetermined period of time back. The content set for parameter items may be unique to the user 202. Therefore, by including the parameter setting history 119 in the query, the generating AI 400 can more easily generate parameters suitable for that user 202, and the likelihood that the parameters presented to the user 202 will be adopted as is can be increased.

[0069] Figure 11 schematically shows an example of the functional configuration of the information processing system 100. The information processing system 100 includes a storage unit 110 and a UI unit 130.

[0070] The memory unit 110 stores various types of information. For example, the memory unit 110 stores skill management information 112. For example, the memory unit 110 stores characteristic information 118. For example, the memory unit 110 stores required parameter item information 117. For example, the memory unit 110 stores dialogue history 114. For example, the memory unit 110 stores call history 115. For example, the memory unit 110 stores parameter setting history 119.

[0071] The UI unit 130 functions as an interface with the user 202. The UI unit 130 includes a presentation unit 132, a user input receiving unit 134, a processing unit 136, a situation recognition unit 138, an operation-related information acquisition unit 140, a query generation unit 142, a skill candidate identification unit 144, a candidate parameter identification unit 146, a setting unit 148, and a learning execution unit 150. It is not mandatory for the UI unit 130 to have all of these.

[0072] The presentation unit 132 makes various presentations to the user 202, and the user input receiving unit 134 receives various inputs from the user 202. The presentation unit 132 may display various information on the screen of the computer 200. The user input receiving unit 134 may acquire user input 111 entered via the input device of the computer 200.

[0073] The processing unit 136 performs various processes. The processing unit 136 performs input and output with the user 202 via the presentation unit 132 and the user input receiving unit 134. The processing unit 136 may execute various applications according to the instructions of the user 202 and perform input and output with the user 202. The processing unit 136 may execute various skills according to the instructions of the user 202 and perform input and output with the user 202.

[0074] The memory unit 110 adds the skills that user 202 has called and used to the call history 115. The memory unit 110 also adds the parameters that user 202 has set in the skill's parameter items to the parameter setting history 119.

[0075] The processing unit 136 may perform an interaction with the user 202. For example, the processing unit 136 executes an AI application and performs an interaction with the user 202 through the AI ​​application. The storage unit 110 adds the content of the interaction with the user 202 to the interaction history 114. For example, the processing unit 136 facilitates the interaction between the generating AI 400 and the user 202 by mediating between the generating AI 400 and the user 202. The storage unit 110 adds the content of the interaction between the generating AI 400 and the user 202 to the interaction history 114.

[0076] The situation recognition unit 138 recognizes the operation status of user 202. The situation recognition unit 138 recognizes, for example, applications that are open on computer 200. The situation recognition unit 138 recognizes, for example, display information 116. The situation recognition unit 138 recognizes, for example, the text content of the information displayed on the screen of computer 200 as display information 116. The situation recognition unit 138 recognizes, for example, the information of the part of the information displayed on the screen of computer 200 that is highly likely to be of attention to user 202 as display information 116. For example, the situation recognition unit 138 recognizes the information selected by user 202 from the information displayed on the screen of computer 200 as display information 116. For example, when multiple windows are displayed on the screen of computer 200, the situation recognition unit 138 recognizes the information contained in the active window as display information 116, the information contained in the top-level window as display information 116, or the information contained in the window where the cursor is located as display information 116.

[0077] The operation-related information acquisition unit 140 acquires operation-related information related to the operations performed by user 202 on the computer 200. For example, the operation-related information acquisition unit 140 acquires user input 111 acquired by the user input reception unit 134 as operation-related information. For example, the operation-related information acquisition unit 140 acquires applications running on the computer 200 recognized by the status recognition unit 138 as operation-related information. For example, the operation-related information acquisition unit 140 acquires display information 116 recognized by the status recognition unit 138 as operation-related information.

[0078] The query generation unit 142 generates queries to be input to the generation AI 400. The query generation unit 142 may generate queries based on the operation-related information acquired by the operation-related information acquisition unit 140 and the skill management information 112.

[0079] The skill candidate identification unit 144 identifies one or more skills as candidates from among multiple skills based on the query generated by the query generation unit 142. The presentation unit 132 may present the one or more skills identified by the skill candidate identification unit 144 to the user 202. The skill candidate identification unit 144 may input the query generated by the query generation unit 142 to the generation AI 400 and identify one or more skills output from the generation AI 400 as candidates.

[0080] The query generation unit 142 may generate queries to implement the skill recommendation function (user-active version). For example, the query generation unit 142 generates a query that includes user input 111 acquired by the user input reception unit 134, skill management information 112, and text indicating the content to output one or more recommended skills from among multiple skills whose skill information is included in the skill management information 112 based on the user input 111. The query generation unit 142 may further generate a query that includes text indicating the content to output the recommendation order if there are multiple recommended skills. The query generation unit 142 may further generate a query that includes at least one of the application information 113, dialogue history 114, and call history 115.

[0081] For example, the query generation unit 142 includes application information 113 in the query. If multiple applications are running on the computer 200, the query generation unit 142 may include application information 113 in the query that includes multiple applications and the hierarchical relationships between them. For example, if multiple applications are running on the computer 200, the query generation unit 142 includes application information 113 in the query that includes information only about the top-level application. For example, if multiple applications are running on the computer 200, the query generation unit 142 includes application information 113 in the query that includes information only about the active application.

[0082] For example, the query generation unit 142 includes the dialogue history 114 in the query. The query generation unit 142 may include the dialogue history 114 from a predetermined period prior to the time the query was generated in the query. The query generation unit 142 may include dialogue trend information generated from the dialogue history 114 from a predetermined period prior to the time the query was generated and from the dialogue history 114 prior to a predetermined period prior to the time the query was generated in the query.

[0083] For example, the query generation unit 142 may include the call history 115 in the query. The query generation unit 142 may include the call history 115, which includes the ranking of skills used by user 202 in the past, in the query.

[0084] The query generation unit 142 may weight multiple application information 113, dialogue history 114, and call history 115 when including them in the query. The query generation unit 142 may apply different weights to multiple application information 113, dialogue history 114, and call history 115. For example, when including application information 113, dialogue history 114, and call history 115 in the query, the query generation unit 142 may apply a first weight, a second weight heavier than the first weight, and a third weight heavier than the second weight. Which weight to apply to each of the application information 113, dialogue history 114, and call history 115 may be set by the administrator of the information processing system 100 or by the user 202. The query generation unit 142 may include text in the query indicating the weights applied to each of the application information 113, dialogue history 114, and call history 115.

[0085] The skill candidate identification unit 144 inputs the query generated by the query generation unit 142 to the generation AI 400, obtains one or more skills output from the generation AI 400, and the presentation unit 132 may present the one or more skills to the user 202 as recommended skill candidates in response to the active user input 111 by the user 202.

[0086] The query generation unit 142 may generate queries to implement the skill recommendation function (user-passive version). For example, the query generation unit 142 generates a query that includes at least one of the applications running on the computer 200 and the display information 116 acquired by the operation-related information acquisition unit 140, skill management information 112, and text indicating the content to output one or more recommended skills from among multiple skills whose skill information is included in the skill management information 112 based on at least one of the applications running on the computer 200 and the display information 116. For example, the query generation unit 142 generates a query that includes the applications running on the computer 200 acquired by the operation-related information acquisition unit 140, skill management information 112, and text indicating the content to output one or more recommended skills from among multiple skills whose skill information is included in the skill management information 112 based on the applications running on the computer 200. For example, the query generation unit 142 generates a query that includes display information 116 acquired by the operation-related information acquisition unit 140, skill management information 112, and text indicating that it will output one or more recommended skills from among multiple skills whose skill information is included in the skill management information 112 based on the display information 116. For example, the query generation unit 142 generates a query that includes the application running on the computer 200 and display information 116 acquired by the operation-related information acquisition unit 140, skill management information 112, and text indicating that it will output one or more recommended skills from among multiple skills whose skill information is included in the skill management information 112 based on the application running on the computer 200 and display information 116. The query generation unit 142 may further generate a query that includes text indicating that it will also output the recommended order if there are multiple recommended skills.

[0087] The query generation unit 142 may include the call history 115 in the query. The query generation unit 142 may include the call history 115, which includes the ranking of skills used by user 202 in the past, in the query.

[0088] The query generation unit 142 may generate queries to implement the parameter generation function. The query generation unit 142 may generate a parameter query to generate candidate parameters, which are candidates for parameters to be set in the target skill, based on the required parameter item information 117 of the target skill for which parameters are to be generated, and the operation-related information acquisition unit 140, which correspond to the user input 111 from user 202. For example, the query generation unit 142 generates a parameter query that includes the user input 111 acquired by the user input reception unit 134, the required parameter item information 117 of the target skill, and text indicating the content to output recommended parameters for the parameter items indicated by the required parameter item information 117 based on the user input 111. If there are multiple recommended parameters, the query generation unit 142 may further generate a query that includes text indicating the content to output the recommended order as well.

[0089] The query generation unit 142 may generate a parameter query that further includes the dialogue history 114. The query generation unit 142 may include the dialogue history 114 from a predetermined period prior to the generation of the parameter query in the parameter query. The query generation unit 142 may include dialogue trend information generated from the dialogue history 114 from a predetermined period prior to the generation of the parameter query and from the dialogue history 114 prior to a predetermined period prior to the generation of the parameter query in the parameter query.

[0090] The query generation unit 142 may generate a parameter query that further includes the characteristic information 118 of the target skill. The query generation unit 142 may also generate a parameter query that further includes the parameter setting history 119 of the target skill.

[0091] The query generation unit 142 may weight multiple elements from the dialogue history 114, characteristic information 118, and parameter setting history 119 when including them in a query. The query generation unit 142 may apply different weights to multiple elements from the dialogue history 114, characteristic information 118, and parameter setting history 119. For example, when including the dialogue history 114, characteristic information 118, and parameter setting history 119 in a query, the query generation unit 142 may apply a first weight, a second weight heavier than the first weight, and a third weight heavier than the second weight. Which weight to apply to each of the dialogue history 114, characteristic information 118, and parameter setting history 119 may be set by the administrator of the information processing system 100 or a user 202, etc. The query generation unit 142 may include text in the query indicating the weights applied to each of the dialogue history 114, characteristic information 118, and parameter setting history 119.

[0092] The candidate parameter identification unit 146 identifies candidate parameters based on the parameter query generated by the query generation unit 142. The candidate parameter identification unit 146 may input the parameter query to the generation AI 400 and identify the parameters output from the generation AI 400 as candidate parameters.

[0093] The setting unit 148 sets the candidate parameters identified by the candidate parameter identification unit 146 to the target skill. The setting unit 148 may present the candidate parameters identified by the candidate parameter identification unit 146 to the user 202 and, upon receiving approval from the user 202, set the candidate parameters to the target skill.

[0094] The query generation unit 142 may generate queries such that the generating AI 400 outputs multiple parameters as candidates for each parameter item. For example, the query generation unit 142 generates a parameter query that includes user input 111 acquired by the user input receiving unit 134, required parameter item information 117 for the target skill, and text indicating the content to output multiple recommended parameters for each of the parameter items indicated by the required parameter item information 117 based on the user input 111. In this case, the setting unit 148 may present multiple parameters to the user 202 for each parameter item and set the parameter selected by the user 202 from among the multiple parameters as the parameter item.

[0095] The learning execution unit 150 performs various types of learning. For example, the learning execution unit 150 performs learning about weighting for input to the generation AI 400. For example, when executing the skill recommendation function (user-active version), the query generation unit 142 generates a query that includes user input 111 and skill management information 112, application information 113 to which the first weight, second weight, and third weight have been applied, dialogue history 114, and call history 115. The skill candidate identification unit 144 inputs the query to the generation AI 400, the presentation unit 132 presents multiple skills output from the generation AI 400 in the recommended order to the user 202, and the processing unit 136 identifies the skill selected by the user 202 and stores it in the storage unit 110. The query generation unit 142 changes the weight application details each time it generates a query. For example, the query generation unit 142 may change the application targets of the first, second, and third weights, or change the magnitude of the values ​​of the first, second, and third weights, each time it generates a query. The learning execution unit 150 performs machine learning using the information stored in the memory unit 110 to identify the weight application content that is most likely to result in the user 202 selecting the skill ranked 1st in recommendation order. After the learning execution unit 150 has identified the weight application content, the query generation unit 142 may apply weights to the application content of the app information 113, the dialogue history 114, and the call history 115.

[0096] Figure 12 schematically shows an example of the processing flow by the information processing system 100. Here, we will explain the processing flow when the information processing system 100 executes the skill recommendation function (user-active version).

[0097] In step 102 (sometimes abbreviated as S), the user input receiving unit 134 obtains user input 111 explicitly entered by user 202. The user input receiving unit 134 obtains user input 111 entered, for example, to the agent window.

[0098] In S104, the query generation unit 142 generates a query. The query generation unit 142 may generate a query that includes user input 111, skill management information 112, at least one of application information 113, dialogue history 114, and call history 115, text indicating that it will output one or more recommended skills from among multiple skills whose skill information is included in the skill management information 112 based on user input 111 and at least one of application information 113, dialogue history 114, and call history 115, and text indicating that if there are multiple recommended skills, it will also output the recommended order.

[0099] In S106, the skill candidate identification unit 144 inputs the query generated by the query generation unit 142 in S104 to the generation AI 400, and identifies one or more skills output from the generation AI 400 as candidate skills.

[0100] In S108, the presentation unit 132 presents the candidate skills identified by the skill candidate identification unit 144 in S106 to the user 202. The presentation unit 132 presents the candidate skills identified by the skill candidate identification unit 144 to the user 202 in a manner that prioritizes them among multiple skills. Prioritizing these skills may, for example, mean placing the candidate skills higher in the order. The presentation unit 132 may also present only the candidate skills identified by the skill candidate identification unit 144 to the user 202 among multiple skills.

[0101] Depending on whether user 202 has selected a skill (YES in S110), the process proceeds to S212, where processing unit 136 executes the selected skill.

[0102] Figure 13 schematically shows an example of the processing flow by the information processing system 100. Here, we will explain the processing flow when the information processing system 100 executes the skill recommendation function (user-passive version).

[0103] In S202, the situation recognition unit 138 recognizes the operation status of user 202 on the computer 200. The situation recognition unit 138 may recognize at least one of the applications running on the computer 200 and the display information 116. If there is a change in user 202's operation status, the process proceeds to S204. For example, if the situation recognition unit 138 recognizes that user 202 has made an input or that the user has made a selection, the process proceeds to S204.

[0104] In S204, the query generation unit 142 generates a query. The query generation unit 142 may generate a query that includes at least one of the applications and display information 116 running on the computer 200 recognized in S202, skill management information 112, text indicating that it will output one or more recommended skills from among multiple skills whose skill information is included in the skill management information 112 based on at least one of the applications and display information 116 running on the computer 200, and text indicating that it will also output the recommended order if there are multiple recommended skills.

[0105] In S206, the skill candidate identification unit 144 inputs the query generated by the query generation unit 142 in S204 to the generation AI 400, and identifies one or more skills output from the generation AI 400 as candidate skills.

[0106] In S208, the presentation unit 132 presents the candidate skills identified by the skill candidate identification unit 144 in S206 to the user 202. The presentation unit 132 presents the candidate skills to the user 202 in a manner that prioritizes the candidate skills identified by the skill candidate identification unit 144 among multiple skills. The presentation unit 132 may also present to the user 202 only the candidate skills identified by the skill candidate identification unit 144 among multiple skills.

[0107] Depending on whether user 202 has selected a skill (YES in S210), the process proceeds to S212, where processing unit 136 executes the selected skill.

[0108] Figure 14 schematically shows an example of the processing flow by the information processing system 100. Here, we will explain the processing flow when the information processing system 100 performs a parameter generation function.

[0109] In S302, the user input receiving unit 134 obtains user input 111 explicitly entered by user 202. The user input receiving unit 134 obtains user input 111 entered to the agent window, for example.

[0110] In S304, the query generation unit 142 generates a query. The query generation unit 142 may generate a query that includes user input 111, skill management information 112, at least one of application information 113, dialogue history 114, and call history 115, text indicating that it will output one or more recommended skills from among multiple skills whose skill information is included in the skill management information 112 based on user input 111 and at least one of application information 113, dialogue history 114, and call history 115, and text indicating that if there are multiple recommended skills, it will also output the recommended order.

[0111] In S306, the skill candidate identification unit 144 inputs the query generated by the query generation unit 142 in S304 to the generation AI 400, and identifies one or more skills output from the generation AI 400 as candidate skills.

[0112] In S308, the presentation unit 132 presents the candidate skills identified by the skill candidate identification unit 144 in S306 to the user 202. The presentation unit 132 presents the candidate skills to the user 202 in a manner that prioritizes the candidate skills identified by the skill candidate identification unit 144 among multiple skills. Prioritizing means, for example, placing the candidate skills higher in the order. The presentation unit 132 may also present only the candidate skills identified by the skill candidate identification unit 144 among multiple skills to the user 202. Depending on whether the user 202 has selected any of the skills (YES in S310), the process proceeds to S312.

[0113] In S312, the query generation unit 142 generates a query to generate candidate parameters for the target skill selected in S310. The query generation unit 142 may generate a query that includes the user input 111 obtained by the user input receiving unit 134 in S302, and if there is user input 111 for the skill selected in S310, that user input 111, the required parameter item information 117 for the target skill, text indicating the content to output recommended parameters for the parameter items indicated by the required parameter item information 117 based on the user input 111, and text indicating the content to output the recommended order if there are multiple recommended parameters.

[0114] In S314, the candidate parameter identification unit 146 inputs the query generated by the query generation unit 142 in S312 to the generation AI 400, and identifies the parameters output from the generation AI 400 as candidate parameters.

[0115] In S316, the presentation unit 132 presents the candidate parameters identified by the candidate parameter identification unit 146 in S314 to the user 202. The presentation unit 132, for example, inputs the candidate parameters into the parameter fields of the target skill in an undetermined state and presents them to the user 202. If there are multiple candidate parameters for a single parameter field, the presentation unit 132 may present multiple candidate parameters as options for that parameter field.

[0116] In response to receiving a skill execution instruction from user 202 using candidate parameters (YES in S318), the process proceeds to S320. In S320, processing unit 136 executes the skill using the parameters for which the execution instruction was received.

[0117] Figure 15 schematically shows an example of the hardware configuration of a computer 1200 that functions as an information processing system 100. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "parts" of the apparatus according to this embodiment, or to cause the computer 1200 to execute operations associated with the apparatus according to this embodiment or such one or more "parts", and / or to cause the computer 1200 to execute a process or a stage of such process according to this embodiment. Such a program may be executed by the CPU 1212 to cause the computer 1200 to execute specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0118] The computer 1200 according to this embodiment includes a CPU 1212, RAM 1214, and a graphics controller 1216, which are interconnected by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive and a DVD-RAM drive, etc. The storage device 1224 may be a hard disk drive and a solid-state drive, etc. The computer 1200 also includes legacy input / output units such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0119] The CPU 1212 operates according to the programs stored in the ROM 1230 and RAM 1214, thereby controlling each unit. The graphics controller 1216 acquires the image data generated by the CPU 1212 and stores it in the frame buffer provided in RAM 1214 or within itself, so that the image data is displayed on the display device 1218.

[0120] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0121] The ROM 1230 stores boot programs and / or hardware-dependent programs of the computer 1200, which are executed by the computer 1200 upon activation. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via USB ports, parallel ports, serial ports, keyboard ports, mouse ports, etc.

[0122] The program is provided on a computer-readable storage medium such as a DVD-ROM or IC card. The program is read from the computer-readable storage medium and installed on a storage device 1224, RAM 1214, or ROM 1230, which are examples of computer-readable storage media, and executed by the CPU 1212. The information processing described within these programs is read by the computer 1200, resulting in coordination between the program and the various types of hardware resources described above. The apparatus or method may be configured to realize the operation or processing of information in accordance with the use of the computer 1200.

[0123] For example, when communication is performed between a computer 1200 and an external device, the CPU 1212 may execute a communication program loaded into RAM 1214 and, based on the processing described in the communication program, instruct the communication interface 1222 to perform communication processing. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in a recording medium such as RAM 1214, storage device 1224, DVD-ROM, or IC card, transmits the read transmission data to the network, or writes received data received from the network to a reception buffer area provided on the recording medium.

[0124] Furthermore, the CPU 1212 may read all or necessary parts of a file or database stored on an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), or an IC card into the RAM 1214, and perform various types of processing on the data in the RAM 1214. The CPU 1212 may then write the processed data back to the external recording medium.

[0125] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and subjected to information processing. The CPU 1212 may perform various types of processing on the data read from RAM 1214, including various types of operations, information processing, conditional judgments, conditional branching, unconditional branching, information retrieval / replacement, etc., as described throughout this disclosure and specified by the program instruction sequence, and write the results back to RAM 1214. The CPU 1212 may also retrieve information in files, databases, etc., within the recording medium. For example, if multiple entries are stored in the recording medium, each having an attribute value of a first attribute associated with an attribute value of a second attribute, the CPU 1212 may search among the multiple entries for an entry that matches the specified condition for the attribute value of the first attribute, read the attribute value of the second attribute stored in that entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies the predetermined condition.

[0126] The program or software module described above may be stored on or near the computer 1200 in a computer-readable storage medium. Alternatively, a recording medium such as a hard disk or RAM provided within a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the program to the computer 1200 via the network.

[0127] In this embodiment, blocks in the flowchart and block diagram may represent a stage in a process in which an operation is performed or a "part" of a device that has the role of performing an operation. A particular stage and "part" may be implemented by a dedicated circuit, a programmable circuit supplied with computer-readable instructions stored on a computer-readable storage medium, and / or a processor supplied with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuit may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuit may include reconfigurable hardware circuits, such as field-programmable gate arrays (FPGAs) and programmable logic arrays (PLAs), which include logical AND, logical OR, exclusive OR, negated AND, negated OR, and other logical operations, flip-flops, registers, and memory elements.

[0128] A computer-readable storage medium may include any tangible device capable of storing instructions that can be executed by a suitable device, and as a result, a computer-readable storage medium having instructions stored therein will comprise a product that includes instructions that can be executed to create means for performing operations specified in a flowchart or block diagram. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disk read-only memory (CD-ROM), digital multipurpose disc (DVD), Blu-ray® disc, memory stick, integrated circuit card, etc.

[0129] Computer-readable instructions may include assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, Java®, C++, and traditional procedural programming languages ​​such as the C programming language or similar programming languages.

[0130] Computer-readable instructions may be provided locally or via a wide area network (WAN) such as a local area network (LAN) or the internet to a processor or programmable circuit of a general-purpose computer, a special-purpose computer, or another programmable data processing device, so that the processor or programmable circuit of the programmable data processing device, such as a computer, may execute the instructions to generate means for performing operations specified in a flowchart or block diagram. Here, the computer may be a PC (personal computer), a tablet computer, a smartphone, a workstation, a server computer, a general-purpose computer, or a special-purpose computer, and may also be a computer system in which multiple computers are connected. Such a computer system in which multiple computers are connected is also called a distributed computing system and is a computer in a broad sense. In a distributed computing system, multiple computers execute a program collectively by each computer executing a part of the program and passing data during program execution between computers as needed.

[0131] Examples of processors include computer processors, central processing units, processing units, microprocessors, digital signal processors, controllers, and microcontrollers. A computer may have one or more processors. In a multiprocessor system with multiple processors, each processor executes a portion of the program, and the processors collectively execute the program by passing program execution data between them as needed. For example, in the execution of multitasks, each of the multiple processors may execute a portion of each task in small chunks by switching tasks at each time slice. In this case, which part of a program each processor executes changes dynamically. Which part of a program each of the multiple processors executes may also be statically determined by multiprocessor-aware programming.

[0132] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications or improvements can be made to the above embodiments. It will be clear from the claims that such modified or improved forms may also be included in the technical scope of the present invention.

[0133] It should be noted that the execution order of operations, procedures, steps, and stages in the apparatus, systems, programs, and methods shown in the claims, specifications, and drawings is not explicitly stated as "before" or "prior to," and that these can be implemented in any order unless the output of a previous process is used in a later process. Even if the operation flow in the claims, specifications, and drawings is described using phrases such as "first," and "next," for convenience, this does not mean that it is essential to perform the operations in that order. [Explanation of symbols]

[0134] 20 Network, 100 Information Processing System, 110 Memory Unit, 111 User Input, 112 Skill Management Information, 113 Application Information, 114 Dialogue History, 115 Call History, 116 Display Information, 117 Required Parameter Item Information, 118 Characteristic Information, 119 Parameter Setting History, 121 Recommended Skills, 122 Recommended Parameters, 130 UI Unit, 132 Presentation Unit, 134 User Input Reception Unit, 136 Processing Unit, 138 Situation Recognition Unit, 140 Operation-Related Information Acquisition Unit, 142 Query Generation Unit, 144 Skill Candidate Identification Unit, 146 Candidate Parameter Identification Unit, 148 Setting Unit, 150 Learning Execution Unit, 200 Computer, 202 User, 212 Mail Application Window, 214 Agent Window, 222 Calendar Application Window, 224 Agent Window, 230 Agent Window, 300 Server, 400 Generative AI, 1200 Computer, 1210 Host Controller, 1212 CPU, 1214 RAM, 1216 Graphics Controller, 1218 Display Device, 1220 Input / Output Controller, 1222 Communication Interface, 1224 Storage Device, 1230 ROM, 1240 Input / Output Chip

Claims

1. A storage unit that stores skill management information, including skill information for multiple skills that constitute a workflow linking multiple types of services, An operation-related information acquisition unit that acquires operation-related information related to the user's operations on a computer operated by the user, A query generation unit that generates queries based on the aforementioned operation-related information and the aforementioned skill management information, A skill candidate identification unit inputs the aforementioned query into a generating AI and identifies one or more skills output from the generating AI as candidates. A presentation unit that presents the one or more skills identified by the skill candidate identification unit to the user. An information processing system equipped with the following features.

2. The operation-related information acquisition unit acquires user input entered by the user as operation-related information, The information processing system according to claim 1, wherein the query generation unit generates a query that includes at least one of the following: an application running on the computer, the history of interaction between the user and the generated AI, and the user's history of using the skill, along with the operation-related information and the skill management information.

3. The information processing system according to claim 2, wherein the query generation unit generates the query including the multiple applications and the hierarchical relationships between the multiple applications when multiple applications are running on the computer.

4. The information processing system according to claim 2, wherein the query generation unit generates a query that includes a dialogue history which

5. The information processing system according to claim 2, wherein the query generation unit generates the query including the usage history, which includes the ranking of the skills used by the user in the past.

6. The operation-related information acquisition unit acquires at least one of the applications running on the computer and the information displayed on the computer screen as operation-related information. The information processing system according to claim 1, wherein the query generation unit generates a query that includes at least one of an application running on the computer and information displayed on the computer's screen, and the skill management information.

7. The information processing system according to claim 6, wherein the query generation unit generates the query which further includes the user's usage history of the skill.

8. The operation-related information acquisition unit acquires user input entered by the user as operation-related information, The query generation unit generates a parameter query for generating candidate parameters, which are candidates for parameters to be set in the target skill, based on the required parameter item information indicating the parameter items required to execute the target skill and the operation-related information. The aforementioned information processing system is A candidate parameter identification unit identifies the candidate parameters based on the parameter query, A setting unit sets the candidate parameters identified by the candidate parameter identification unit to the target skill. An information processing system according to any one of claims 1 to 7, comprising:

9. The information processing system according to claim 8, wherein the query generation unit generates a parameter query that further includes at least one of characteristic information indicating the characteristics of the target skill, the history of conversations with the user, and the history of parameters previously set for the target skill.

10. The information processing system according to any one of claims 1 to 7, further comprising the aforementioned generating AI.

11. A method of information processing performed by a computer, The storage stage involves storing skill management information, which includes skill information for multiple skills that constitute a workflow linking multiple types of services, in the storage unit. An operation-related information acquisition step is to acquire operation-related information related to the user's operations on the computer operated by the user, A query generation step in which a query is generated based on the aforementioned operation-related information and the aforementioned skill management information, Based on the aforementioned query, the identification step involves identifying one or more skills as candidates from the multiple skills, A presentation stage in which the one or more skills identified in the aforementioned specific stage are presented to the user as candidates. An information processing method comprising the following:

12. A program for causing a computer to execute the information processing method described in claim 11.

Images