Information processing system, information processing method, and program

Abstract

To provide an information processing system, an information processing method, and a program.SOLUTION: An information processing system in which a plurality of computers and a server communicate with each other via a network includes, as a functional configuration, a storage unit that stores skill management information including skill information of a plurality of skills constituting a workflow in which a plurality of types of services cooperate with each other, an operation-related information acquiring unit that acquires operation-related information related to an operation of a user in a computer operated by the user, a query generation unit that generates a query based on the operation-related information and the skill management information, and an UI unit that includes a skill candidate specifying unit that specifies one or a plurality of skills output from the generation AI as candidates by inputting the query to the generation AI and a presentation unit that presents the one or the plurality of skills specified by the skill candidate specifying unit to the user.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] Patent Document 1 describes an intelligent user assistance function that supports software users by inferring their needs and preferences in operating a software system or application. Patent Document 2 describes a workflow management system that can easily reuse past workflow components and present related information in a timely manner. Patent Document 3 describes AI (Artificial Intelligence) that uses computer vision to recognize apps, screens, and UI (User Interface) elements. Patent Document 4 describes a business process generation system for generating business processes used in a network environment using PaaS (Platform as a Service). [Prior art document] [Patent documents] [Patent Document 1] Special Publication No. 2001-510599 [Patent Document 2] JP 2007-188145 A [Patent Document 3] Special Publication No. 2023-545253 [Patent Document 4] JP 2012-242954 A Summary of the Invention [Means for solving the problem]

[0003] According to one embodiment of the present invention, an information processing system is provided. The information processing system may include a storage unit that stores skill management information including skill information for multiple skills that constitute a workflow that links multiple types of services. The information processing system may include an operation-related information acquisition unit that acquires operation-related information related to a user's operation on a computer operated by the user. The information processing system may include a query generation unit that generates a query based on the operation-related information and the skill management information. The information processing system may include a skill candidate identification unit that inputs the query to a generation AI and identifies one or more skills output from the generation AI as candidates. The information processing system may include a presentation unit that presents the one or more skills identified by the skill candidate identification unit to the user.

[0004] In the information processing system, the operation-related information acquisition unit may acquire user input entered by the user as the operation-related information, and the query generation unit may generate the query including at least one of applications running on the computer, a dialogue history between the user and the generation AI, and a usage history of the skill by the user, the operation-related information, and the skill management information. In any of the information processing systems, the query generation unit may, when multiple applications are running on the computer, generate the query including app information including the multiple applications and a hierarchical relationship between the multiple applications. In any of the information processing systems, the query generation unit may generate the query including the dialogue history including a dialogue history with the user from a predetermined period going back from the time of generation of the query, and dialogue tendency information generated from a dialogue history with the user before the predetermined period going back from the time of generation of the query. In any of the information processing systems, the query generation unit may generate the query including the usage history including a usage ranking of the skills previously used by the user.

[0005] In any of the information processing systems, the operation-related information acquisition unit may acquire at least one of an application running on the computer and information displayed on a screen of the computer as the operation-related information, and the query generation unit may generate the query including at least one of the application running on the computer and the information displayed on a screen of the computer, and the skill management information. The query generation unit may generate the query further including a usage history of the skill by the user.

[0006] In any of the information processing systems, the operation-related information acquisition unit may acquire user input entered by the user as the operation-related information, and the query generation unit may generate a parameter query for generating candidate parameters that are candidates for parameters to be set in the target skill based on required parameter item information indicating parameter items required to execute the target skill and the operation-related information, and the information processing system may include: a candidate parameter identification unit that identifies the candidate parameters based on the parameter query; and a setting unit that sets the candidate parameters identified by the candidate parameter identification unit to the target skill. The query generation unit may generate the parameter query further including at least one of characteristic information indicating characteristics of the target skill, a dialogue history with the user, and a history of parameters previously set for the target skill.

[0007] Any of the information processing systems may further include the generation AI.

[0008] According to one embodiment of the present invention, there is provided an information processing method executed by a computer. The information processing method may include a storage step of storing, in a storage unit, skill management information including skill information of a plurality of skills that constitute a workflow linking a plurality of types of services. The information processing method may include an operation-related information acquisition step of acquiring operation-related information related to a user's operation on a computer operated by the user. The information processing method may include a query generation step of generating a query based on the operation-related information and the skill management information. The information processing method may include an identification step of identifying one or more skills from the plurality of skills as candidates based on the query. The information processing method may include a presentation step of presenting the one or more skills identified in the identification step to the user as candidates.

[0009] According to one embodiment of the present invention, there is provided a program for causing a computer to execute the information processing method.

[0010] The above summary of the invention does not list all of the necessary features of the present invention, and subcombinations of these features may also constitute inventions. [Brief explanation of the drawings]

[0011] [Figure 1] 1 illustrates an example of an information processing system 100. [Figure 2] FIG. 2 is an explanatory diagram for explaining a skill recommendation function (user active version) of the information processing system 100. [Figure 3] FIG. 2 is an explanatory diagram for explaining a skill recommendation function (user active version) of the information processing system 100. [Figure 4] FIG. 2 is an explanatory diagram for explaining a skill recommendation function (user passive version) of the information processing system 100. [Figure 5] FIG. 2 is an explanatory diagram for explaining a parameter generation function when a skill is executed by the information processing system 100. [Figure 6] FIG. 2 is an explanatory diagram for explaining a parameter generation function when a skill is executed by the information processing system 100. [Figure 7] FIG. 2 is an explanatory diagram for explaining a parameter generation function when a skill is executed by the information processing system 100. [Figure 8] 10 is an explanatory diagram illustrating an example of input and output to a generated AI 400 by the information processing system 100. FIG. [Figure 9] 10 is an explanatory diagram illustrating an example of input and output to a generated AI 400 by the information processing system 100. FIG. [Figure 10] 10 is an explanatory diagram illustrating an example of input and output to a generated AI 400 by the information processing system 100. FIG. [Figure 11] 1 illustrates an example of a functional configuration of an information processing system 100. [Figure 12] 10 shows an example of a processing flow by the information processing system 100. [Figure 13] 10 shows an example of a processing flow by the information processing system 100. [Figure 14] 10 shows an example of a processing flow by the information processing system 100. [Figure 15] 1 shows an example of a hardware configuration of a computer 1200 that functions as the information processing system 100. DETAILED DESCRIPTION OF THE INVENTION

[0012] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention according to the claims. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0013] Previously, generative AI services such as ChatGPT (registered trademark) could accept user prompts and provide answers tailored to the user's questions, but they struggled to understand the user's situation and the challenges they faced and provide thoughtful suggestions and answers. Furthermore, users had to manually write prompts and master the generative AI. In other words, only users with the ability to master generative AI (prompt engineering skills) could fully utilize its power, preventing the "ultra" efficiency of business operations achieved by generative AI. The information processing system 100 according to this embodiment may function as a business support platform. For example, a user can select a section of interest on a SaaS application screen, and the generative AI will interpret the business issues from the current situation, past usage history, etc., and suggest appropriate apps and custom workflows. The selection of a section of text may include range text selection, which is the act of selecting a range of text with an operation similar to that used when copying text. The strip selection may include area screenshot format selection, which is the act of selecting an area and taking a screenshot by dragging from the upper left fulcrum to the lower right fulcrum, with an operation feel similar to the function in Windows (registered trademark) that specifies an area to take a screenshot, which is assigned to "Windows" + "Shift" + "S," or the function in Mac (registered trademark) that specifies an area to take a screenshot, which is assigned to "Shift" + "Command" + "4."

[0014] 1 schematically illustrates an example of an information processing system 100. The information processing system 100 may be realized by a computer 200 operated by a user 202. The information processing system 100 realized by the computer 200 supports the operation of the computer 200 by the user 202.

[0015] In a configuration in which the computer 200 accesses the server 300 to execute various processes, the information processing system 100 may be realized by the server 300. For example, in a thin client environment, most of the functions are arranged in the server 300, and the computer 200 executes limited functions such as input and display, but the information processing system 100 realized in the server 300 supports the user 202 in operating the computer 200 to execute processes in the server 300. The computer 200 and the server 300 may communicate via a network 20. The network 20 may include a cloud. The network 20 may include the Internet. The network 20 may include a mobile communication network. The network 20 may include a LAN (Local Area Network).

[0016] The information processing system 100 has multiple skills available to the user 202. The skills may constitute a workflow that links multiple types of services. The services may be so-called SaaS (Software as a Service). The services may be services provided by applications. A different workflow is configured for each skill. For example, a certain skill constitutes a workflow that links a web conferencing app that provides a web conferencing service, an AI app that provides a service that transcribes web conference audio and creates minutes, an AI app that provides a summarization service that summarizes the minutes, and a CRM app that provides a CRM service that stores and manages meeting and summary minutes. A workflow may link any number of services. A workflow may link any type of service. The information processing system 100 may create and manage multiple skills using so-called iPaaS (Integration Platform as a Service).

[0017] The information processing system 100 stores skill management information including information on multiple skills (sometimes referred to as skill information). The skill information may include skill identification information that can identify the skill. The skill identification information may be, for example, the name of the skill. The skill information may include a description that explains the skill. For example, the skill information of a skill that constitutes a workflow that links a web conference app, an AI application that provides a service that transcribes the audio of a web conference and creates minutes, an AI application that provides a summarization service that summarizes the minutes, and a CRM application that provides a CRM service that saves and manages the conference and summary minutes includes a description that explains that the skill provides the web conference service, the service that transcribes the audio of a web conference and creates minutes, the summarization service that summarizes the minutes, and the CRM service that saves and manages the conference and summary minutes. The skill information may include information on each of multiple types of services (sometimes referred to as service information) that the workflow constituted by the skill links with. The service information may include service identification information that can identify the service. The service identification information may be, for example, the name of the service. The service information may include the type of service. The service information may include a description that describes the service. The service information may include information about an application that realizes the service. The application information may include app identification information that can identify the application. The app identification information may be the name of the application. The application information may include the type of application. The application information may include a description that describes the application.

[0018] Note that a skill may correspond to an application. That is, the information processing system 100 may have a plurality of skills that each constitute a workflow and a plurality of skills that each constitute an application, and the skill management information may include skill information for the plurality of skills that each constitute a workflow and skill information for the plurality of skills that each constitute an application.

[0019] For example, in response to an input by the user 202 (sometimes referred to as a user input), the information processing system 100 identifies one or more recommended skills from a plurality of skills and presents them as candidates to the user 202. For example, the information processing system 100 presents the identified plurality of skills to the user 202 in the order of recommendation.

[0020] For example, without explicit input from the user 202, the information processing system 100 identifies one or more recommended skills from among multiple skills based on open apps, information displayed on the screen, etc., and presents the skills as candidates to the user 202. For example, the information processing system 100 presents the identified multiple skills to the user 202 in the order of recommendation.

[0021] The information processing system 100 may use the generated AI 400 to assist the user 202 in operating the computer 200. The generated AI 400 may be a large language model (LLM). The generated AI 400 may be a so-called interactive generated AI. The generated AI 400 may be a generated AI that combines a large language model (LLM) with an interactive UI. Examples of the generated AI 400 include ChatGPT, Gemini, and Claude (registered trademark), but are not limited to these. The generated AI 400 may be any existing generated AI. The generated AI 400 may also be a generated AI generated for the information processing system 100. The information processing system 100 may communicate with the generated AI 400 via the network 20. The information processing system 100 may also be provided with the generated AI 400.

[0022] Highly accurate output can be obtained by appropriately inputting necessary information to the generation AI 400. The information processing system 100 may generate a query to input to the generation AI 400, which will enable the generation AI 400 to understand the situation or problem the user 202 is facing and select a skill recommended for that situation or problem.

[0023] For example, the information processing system 100 generates a query including an instruction to select a skill recommended for the operation-related information from a plurality of skills whose skill information is included in the skill management information, based on operation-related information related to the operation of the user 202 on the computer 200 operated by the user 202 and skill management information of a plurality of skills possessed by the information processing system 100.

[0024] The operation-related information may be a user input or may be information indicating the status of computer 200, such as an application running on computer 200 or information displayed on the screen of computer 200.

[0025] The information processing system 100 generates a query including, for example, operation-related information, skill management information, and text indicating instructions for outputting one or more recommended skills from among a plurality of skills whose skill information is included in the skill management information based on the operation-related information, and inputs the query to the generation AI 400. The information processing system 100 presents the one or more skills output from the generation AI 400 to the user 202. This may allow candidate skills suitable for the situation in which the user 202 finds himself or the problem he or she is facing to be presented to the user 202.

[0026] When an existing generation AI receives an input requesting recommendations for applications to achieve a certain goal, it outputs a response including the recommended applications. This is because the existing generation AI acquires and learns information about various applications from the web, etc. On the other hand, existing generation AIs have not yet realized the ability to select and recommend one or more skills from multiple skills that constitute a workflow implemented on a specific computer 200. In contrast, for example, the information processing system 100 according to the present embodiment has a function for generating and managing multiple skills that constitute a workflow, or a function for managing multiple generated skills, using iPaaS, etc., and further has a function for including skill management information in a query input to the generation AI 400. This allows the generation AI 400 to select and output recommended skills for operation-related information and present them to the user 202. The number of workflow variations is proportional to the number of combinations of linked services, which can be extremely large compared to the number of applications, etc. In other words, the number of skills can be extremely large. While mastering a huge number of skills would greatly improve convenience for the user 202, the hurdle to mastering them would be extremely high. In contrast, the information processing system 100 can select recommended skills for operation-related information from a huge number of skills and present them to the user 202, thereby supporting the user 202 in mastering the skills and greatly contributing to improving convenience for the user 202.

[0027] 2 and 3 are explanatory diagrams for explaining the skill recommendation function (user active version) of the information processing system 100. The information processing system 100 presents recommended skills in response to an explicit user input by the user 202.

[0028] 2 and 3, an email application window 212, which is a window for an email application, and an agent window 214, which is a window for an agent for assisting the user 202, are displayed. Fig. 2 shows a state in which no user input has been made to the agent window 214, and the agent window 214 is in a default display state. In the default state of the agent window 214, multiple skills may be displayed arranged in a default order.

[0029] FIG. 3 shows the state of the agent window 214 when the same content as an email message is entered into the agent window 214. For example, in response to a user input, the information processing system 100 generates a query including the entered email message, skill management information, and information indicating that the application being used is an email application, inputs the query to the generation AI 400, and acquires one or more skills output from the generation AI 400. In FIG. 3, the generation AI 400 outputs a skill called "automatic generation of approval documents" recommended for the content of the email message, and the information processing system 100 presents the multiple skills in order of priority, with "automatic generation of approval documents" as the priority. This allows the user 202 to proactively be notified of the skills they should use for the email message, thereby improving user convenience.

[0030] 4 is an explanatory diagram for explaining the skill recommendation function (user-passive version) of the information processing system 100. The information processing system 100 identifies recommended skills from the status of the computer 200 and presents them to the user 202 in a situation where there is no explicit input from the user 202.

[0031] In the example shown in FIG. 4, a calendar application window 222, which is a window for a calendar application, is displayed, and shows a state in which the user 202 has selected an appointment titled "Business Negotiation with Company A" registered in the calendar. In response to the user 202 selecting "Business Negotiation with Company A," the information processing system 100 generates a query including the content of the selected appointment, skill management information, and information that the application being used is a calendar application, inputs the query to the generation AI 400, and acquires one or more skills output from the generation AI 400. In FIG. 4, the generation AI 400 outputs a skill called "summarize minutes and save in CRM," a skill called "create business negotiation slides," and a skill called "contact to arrange a date from an available slot" that are recommended for the appointment titled "Business Negotiation with Company A," and the information processing system 100 presents an agent window 224 that includes these skills. This allows the user 202 to be proactively notified of recommended skills according to the user's operation status, thereby improving the user's convenience.

[0032] As described above, the information processing system 100 can present to the user 202 skill candidates that are suitable for the situation the user 202 is in or the problem the user is facing, and the user 202 can select a desired skill from the presented candidate skills and proceed with using the skill. Some skills begin to be executed upon selection, while others may not begin to be executed unless the user 202 inputs minimum required parameters. In response to this, the information processing system 100 may further support the input of minimum required parameters for a skill.

[0033] 5, 6, and 7 are explanatory diagrams for explaining the parameter generation function when a skill is executed by the information processing system 100. When a skill is executed, the information processing system 100 may identify and set minimum required parameters based on user input and the status of the computer 200.

[0034] The example shown in FIG. 5 shows a state in which the user 202 inputs "I need to buy some vegetables" into the agent window 230, and the information processing system 100 presents a number of recommended skills. FIG. 6 shows a state in which the user 202 selects the skill "Ask the AI." The information processing system 100 inputs the user input "I need to buy some vegetables" into the generated AI 400, and presents the output from the generated AI 400 in the agent window 230. In the example shown in FIG. 6, the message presented is "Which vegetables do you plan to buy? Also, let us know if you have any cooking plans. We can also give you recommendations on how to use them."

[0035] FIG. 7 illustrates a state in which the skill "Add a task to task management" has been selected by the user 202. Here, it is assumed that the minimum parameter items required to execute the skill "Add a task to task management" are the task name and details. The information processing system 100 generates a parameter query for generating recommended parameters to be set in the skill "Add a task to task management" based on a user input such as "I need to buy some vegetables," display information such as "Which vegetables do you plan to buy? Also, please let me know if you have any cooking plans. I can also give you some recommendations on how to use them," and the required parameter items, namely, the task name and details. The information processing system 100 generates a parameter query including, for example, the user input, the display information, required parameter item information indicating the required parameter items, and an instruction to output recommended parameters for the required parameter items indicated by the required parameter item information based on the user input and the display information. The information processing system 100 inputs the generated parameter query to the generation AI 400, obtains the parameters output by the generation AI 400, and sets them in the skill "Add a task to task management." 7, the parameter "task name" is set to "purchase vegetables," and the parameter "details" is set to "purchase necessary vegetables at the supermarket." This can support the use of skills by the user 202, and can contribute to improving the operating environment for the user 202.

[0036] 8 is an explanatory diagram for explaining an example of input and output to the generation AI 400 by the information processing system 100. Here, input and output to the generation AI 400 by the information processing system 100 to realize the skill recommendation function (user active version) will be described.

[0037] The information processing system 100 inputs to the generation AI 400 a query that the generation AI 400 has generated based on a user input 111 so as to output recommended skills from among a plurality of skills whose skill information is included in the skill management information 112, and acquires recommended skills 121 from the generation AI 400. The information processing system 100 may input to the generation AI 400, for example, a query that includes the user input 111, the skill management information 112, and text indicating content to output one or more recommended skills from among a plurality of skills whose skill information is included in the skill management information 112 based on the user input 111. The information processing system 100 may input to the generation AI 400 a query that further includes text indicating content to output a recommendation order when there are a plurality of recommended skills.

[0038] The user input 111 may be an image. For example, the user input 111 may be an image generated by taking a screenshot of part or all of the screen of the computer 200. The user input 111 may be an image of an area of ​​the screen of the computer 200 designated by the user 202. The user input 111 may be an image selected by a strip. The user input 111 may be an image selected by a range screenshot selection. The user input 111 may be text. The user input 111 may be text selected by a range text selection. The user input 111 may be audio. The user input 111 may include other types of input by the user 202.

[0039] The skill management information 112 may include skill information of a plurality of skills that the information processing system 100 possesses and that can be used by the user 202. The information processing system 100 may include the skill management information 112 in a query as a base of skills to be selected by the generation AI 400.

[0040] The information processing system 100 may generate a query that further includes app information 113 open on the computer 200. The app information 113 may be information about applications running on the computer 200. The application information may include app identification information that identifies the application. The application information may include app type information that indicates the type of application. Various types of applications are known, including email applications, calendar applications, web conferencing applications, and business management applications. In this embodiment, the types of applications may include all known types or only a portion of all known types. It is believed that there is a strong correlation between applications running on the computer 200 and the interests of the user 202. Therefore, by inputting the app information 113 into the generation AI 400 in addition to the user input 111 and the skill management information 112, the generation AI 400 may be more likely to select skills that are more suitable for the interests of the user 202 than if the app information 113 were not input. In other words, by inputting the app information 113 into the generation AI 400, it may be possible to recommend skills that are more suitable for the user 202 at that time.

[0041] If multiple applications are running on the computer 200, the information processing system 100 may include app information 113 containing information about the multiple applications in the query. In this case, the app information 113 may include the hierarchical relationship between the multiple applications. For example, the app information 113 includes the hierarchical order of the windows of the multiple applications. An application located at a shallower level in the hierarchy is more likely to attract the user 202's interest than an application located at a deeper level in the hierarchy. For example, if a calendar application is located at the top, a business management application is located below that, and an email application is located below that, the user 202 is most likely to be most interested in the calendar, i.e., schedule, followed by business management, and then email. Therefore, inputting app information 113 containing the hierarchical relationship between the multiple applications into the generation AI 400 may increase the likelihood that the generation AI 400 will select skills appropriate for the user 202's multiple interests and the levels of those interests, compared to when the app information 113 is not input. When multiple applications are running on the computer 200, the information processing system 100 may include, in the query, app information 113 including information on only the application located at the top level. When multiple applications are running on the computer 200, the information processing system 100 may include, in the query, app information 113 including information on only the application that is active.

[0042] The app information 113 may include a total usage time, which is the total amount of time the application has been used since the application began execution. When multiple applications are running on the computer 200, the app information 113 may include a total usage time for each of the multiple applications. An application with a longer total usage time is more likely to attract the user 202's interest than an application with a shorter total usage time. Therefore, inputting app information 113 including the total usage time of multiple applications into the generation AI 400 may increase the likelihood that the generation AI 400 will select skills that are appropriate for the user 202's multiple interests and the level of those interests, compared to when the app information 113 is not input.

[0043] The application information 113 may include the elapsed time since the application started to run. When multiple applications are running on the computer 200, the application information 113 may include the elapsed time for each of the multiple applications. An application with a shorter elapsed time is more likely to attract the user 202's interest than an application with a longer elapsed time. Therefore, inputting application information 113 including the elapsed times for multiple applications into the generation AI 400 may increase the likelihood that the generation AI 400 will select skills that are appropriate for the user 202's multiple interests and the level of those interests, compared to when the application information 113 is not input.

[0044] The application information 113 may include the window size of the application. When multiple applications are running on the computer 200, the application information 113 may include the window size of each of the multiple applications. An application with a large window size is more likely to attract the user 202's interest than an application with a small window size. Therefore, inputting application information 113 including the window sizes of multiple applications into the generation AI 400 may increase the likelihood that the generation AI 400 will select skills that are appropriate for the user 202's multiple interests and the level of those interests, compared to when the application information 113 is not input.

[0045] The information processing system 100 may generate a query that further includes a past interaction history 114. The interaction history 114 is a history of past interactions with the user 202. The information processing system 100 may accumulate a history of interactions with the user 202 and may include the interaction history 114 in the query. The interaction history 114 may be a history of interactions between the information processing system 100 and the user 202. In other words, the interaction history 114 may be a history of input and output between the information processing system 100 and the user 202. The interaction history 114 may be a history of interactions between the user 202 and the generated AI 400 mediated by the information processing system 100. The interaction history 114 includes information such as what the user 202 wanted from the information processing system 100, what the user 202 thought about the output from the information processing system 100, what the user 202 wanted from the generated AI 400, and how the user 202 felt about the output from the generated AI 400. Therefore, by inputting the dialogue history 114 into the generation AI 400, it may be possible to increase the likelihood that the generation AI 400 will select a skill that is in line with the user 202's wishes, or to reduce the likelihood that the generation AI 400 will select a skill that is contrary to the user 202's wishes.

[0046] The information processing system 100 may include in the query the interaction history 114 from a predetermined period back. Since older interaction history 114 may not match the interests of the current user 202, inputting only newer interaction history 114 to the generation AI 400 may increase the likelihood that the generation AI 400 will select skills that are more suited to the interests of the current user 202.

[0047] The information processing system 100 may include in a query the dialogue history 114 from a predetermined period of time back and dialogue tendency information generated from the dialogue history 114 from before the predetermined period of time back. The dialogue tendency information may include information summarizing the dialogue history 114. The dialogue tendency information may include information on characteristic topics extracted from the dialogue history 114. The dialogue tendency information may include frequently appearing topics extracted from the dialogue history 114. Compared to the new dialogue history 114, the old dialogue history 114 is less likely to be relevant to the interests of the current user 202, but may still be relevant to the interests of the current user 202. Therefore, by inputting the new dialogue history 114 as is and inputting the old dialogue history 114 as dialogue tendency information, the old dialogue history 114 can also be appropriately utilized, which may increase the likelihood that the generation AI 400 will select skills appropriate for the interests of the current user 202.

[0048] The information processing system 100 may generate a query that further includes a past skill invocation history 115. The invocation history 115 is a usage history of skills that the user 202 has invoked and used in the past. The information processing system 100 may accumulate the skill invocation history 115 by the user 202 and may include the invocation history 115 in the query. The invocation history 115 includes information on skills that the user 202 frequently uses among multiple skills. Therefore, by inputting the invocation history 115 into the generation AI 400, it may be possible to increase the likelihood that the generation AI 400 will select a skill that meets the user 202's desires.

[0049] The call history 115 may include, for each skill used by the user 202, information that can identify the skill and the timing at which the skill was used. The timing at which the skill was used may be a date or a time and date. The call history 115 may include the number of times the skill was used by the user 202. The call history 115 may include the order in which the skills used by the user 202 were used. The call history 115 may include skills that have been used by the user 202 among multiple skills. In other words, the call history 115 includes skills that have been used by the user 202, but does not include skills that have not been used by the user 202. By including these in the call history 115, not only the skills used by the user 202 but also the skill usage tendency of the user 202 can be input to the generation AI 400, which may increase the likelihood that the generation AI 400 will select skills that meet the user 202's desires.

[0050] 9 is an explanatory diagram for explaining an example of input and output to the generation AI 400 by the information processing system 100. Here, input and output to the generation AI 400 by the information processing system 100 to realize the skill recommendation function (user passive version) will be described.

[0051] The information processing system 100 inputs to the generation AI 400 a query generated by the generation AI 400 to output recommended skills from among a plurality of skills whose skill information is included in the skill management information 112 based on the application information 113 open on the computer 200 and the display information 116 displayed on the screen of the computer 200, and acquires recommended skills 121 from the generation AI 400. The information processing system 100 may input to the generation AI 400 a query including, for example, the application information 113, the display information 116, the skill management information 112, and text indicating content to output one or more recommended skills from among a plurality of skills whose skill information is included in the skill management information 112 based on the application information 113 and the display information 116. The information processing system 100 may input to the generation AI 400 a query further including text indicating content to output a recommendation order when there are a plurality of recommended skills.

[0052] The information processing system 100 may input the display information 116 to the generation AI 400 only if approval for inputting the display information 116 to the generation AI 400 has been obtained from the user 202. If approval for inputting the display information 116 to the generation AI 400 has not been obtained from the user 202, the information processing system 100 generates a query that does not include the display information 116 and inputs it to the generation AI 400.

[0053] The app information 113 may be information about an application running on the computer 200. In the absence of explicit input from the user 202, the application running on the computer 200 can be useful information for estimating the interests of the user 202. Therefore, inputting the app information 113 into the generation AI 400 may enable the generation AI 400 to select skills that are suitable for the interests of the user 202.

[0054] When multiple applications are running on the computer 200, the information processing system 100 may include app information 113 including information about the multiple applications in the query. In this case, the app information 113 may include the hierarchical relationship between the multiple applications. For example, the app information 113 includes the hierarchical order of windows of the multiple applications. Note that when multiple applications are running on the computer 200, the information processing system 100 may include app information 113 including information about only the application located at the top level in the query. When multiple applications are running on the computer 200, the information processing system 100 may include app information 113 including information about only the active application in the query.

[0055] The application information 113 may include a total usage time, which is the total time that the application has been used since the application started to be executed. If multiple applications are running on the computer 200, the application information 113 may include a total usage time for each of the multiple applications.

[0056] The application information 113 may include the elapsed time since the execution of the application started. If multiple applications are running on the computer 200, the application information 113 may include the elapsed time for each of the multiple applications.

[0057] The application information 113 may include the window size of the application. If multiple applications are running on the computer 200, the application information 113 may include the window size of each of the multiple applications.

[0058] The display information 116 may include all information displayed on the screen of the computer 200. The display information 116 may include a portion of all information displayed on the screen of the computer 200. The display information 116 may include text from all information displayed on the screen of the computer 200. The information processing system 100, for example, includes in a query, as the display information 116, a portion of information displayed on the screen of the computer 200 that is likely to be of interest to the user 202. For example, the information processing system 100 includes in a query, among the information displayed on the screen of the computer 200, information selected by the user 202. For example, when multiple windows are displayed on the screen of the computer 200, the information processing system 100 may include in a query information included in an active window, information included in a top-level window, or information included in a window where the cursor or the like is located. In a situation where there is no explicit input from the user 202, the display information 116 can be useful information for estimating the user 202's interests. Thus, with the user's 202 approval, the display information 116 may be input to the generation AI 400 to enable the generation AI 400 to select skills that are appropriate for the user's 202 interests.

[0059] The skill management information 112 may include skill information of a plurality of skills that the information processing system 100 possesses and that can be used by the user 202. The information processing system 100 may include the skill management information 112 in a query as a base of skills to be selected by the generation AI 400.

[0060] The information processing system 100 may generate a query that further includes a skill invocation history 115. The invocation history 115 is a usage history of skills that have been invoked and used in the past by the user 202. The information processing system 100 may accumulate a skill usage history by the user 202 and may include the invocation history 115 in the query.

[0061] The call history 115 may include, for each skill used by the user 202, information that can identify the skill and the timing at which the skill was used. The timing at which the skill was used may be a date or a time and date. The call history 115 may include the number of times the skill was used by the user 202. The call history 115 may include the order of the number of times the skill was used by the user 202. The call history 115 may include skills that have been used by the user 202 out of multiple skills. In other words, the call history 115 includes skills that have been used by the user 202 and does not include skills that have not been used by the user 202.

[0062] The information processing system 100 may generate a query that further includes a dialogue history, which is a history of past dialogues with the user 202. The information processing system 100 may accumulate a history of dialogues with the user 202 and include the dialogue history in the query. The dialogue history may be a dialogue history between the information processing system 100 and the user 202. In other words, the dialogue history may be a history of input and output between the information processing system 100 and the user 202. The dialogue history may be a history of dialogues between the user 202 and the generation AI 400 mediated by the information processing system 100. The information processing system 100 may include a dialogue history from a predetermined period back in the query. The information processing system 100 may include a dialogue history from a predetermined period back in the query and dialogue tendency information generated from a dialogue history from a predetermined period back in the query and an dialogue history before the predetermined period back in the query. The dialogue tendency information may include information summarizing the dialogue history. The dialogue tendency information may include information on characteristic topics extracted from the dialogue history. The dialogue tendency information may include topics that appear frequently in the dialogue history, extracted from the dialogue history.

[0063] 10 is an explanatory diagram for explaining an example of input and output to the generation AI 400 by the information processing system 100. Here, input and output to the generation AI 400 by the information processing system 100 for realizing the parameter generation function will be described.

[0064] The information processing system 100 inputs to the generation AI 400 a query that the generation AI 400 has generated based on the user input 111 so as to output parameters recommended for the parameter items indicated by the required parameter item information 117, and acquires recommended parameters 122 from the generation AI 400. The information processing system 100 may input to the generation AI 400, for example, a query that includes the user input 111, the required parameter item information 117, and text indicating content for outputting parameters recommended for the parameter items indicated by the required parameter item information 117 based on the user input 111. The information processing system 100 may input to the generation AI 400 a query that further includes text indicating content for outputting the order of recommendation when there are multiple recommended parameters.

[0065] The user input 111 may be an image. For example, the user input 111 may be an image generated by taking a screenshot of part or all of the screen of the computer 200. The user input 111 may be an image of an area of ​​the screen of the computer 200 designated by the user 202. The user input 111 may be an image selected by a strip. The user input 111 may be an image selected by a range screenshot selection. The user input 111 may be text. The user input 111 may be text selected by a range text selection. The user input 111 may be audio. The user input 111 may include other types of input by the user 202.

[0066] The information processing system 100 may generate a query that further includes a past dialogue history 114. The dialogue history 114 is a history of past dialogues with the user 202. The information processing system 100 may accumulate a history of dialogues with the user 202 and may include the dialogue history 114 in the query. The dialogue history 114 may be a history of dialogues between the information processing system 100 and the user 202. In other words, the dialogue history 114 may be a history of input and output between the information processing system 100 and the user 202. The dialogue history 114 may be a history of dialogues between the user 202 and the generated AI 400 mediated by the information processing system 100. The information processing system 100 may include the dialogue history 114 from a predetermined period back in the query. The information processing system 100 may include the dialogue history 114 from a predetermined period back in the query and dialogue tendency information generated from the dialogue history 114 from a predetermined period back in the query. The dialogue tendency information may include information summarizing the dialogue history 114. The dialogue tendency information may include information on characteristic topics extracted from the dialogue history 114. The dialogue tendency information may include topics that appear frequently in the dialogue history 114, extracted from the dialogue history 114.

[0067] The information processing system 100 may generate a query that further includes characteristic information 118. The characteristic information 118 indicates characteristics of a target skill. The characteristic information 118 may indicate characteristics of multiple services that are linked by a workflow configured by the target skill. The characteristic information 118 may indicate characteristics of each app of the multiple services. For example, if the target skill configures a workflow that links a web conference app that provides a web conference service, an AI app that provides a service that transcribes web conference audio and creates minutes, an AI app that provides a summarization service that summarizes the minutes, and a CRM app that provides a CRM service that stores and manages conference and summarized minutes, the characteristic information 118 may include web conference, transcription, summarization, and storage. Including the characteristic information 118 in the query in addition to the required parameter item information 117 can make it easier for the generation AI 400 to generate parameters corresponding to the characteristics for multiple parameter items of the target skill, which may increase the likelihood that the parameters presented to the user 202 will be adopted as is by the user 202.

[0068] The information processing system 100 may generate a query that further includes a parameter setting history 119. The parameter setting history 119 is a history of parameters previously set for parameter items indicated by the required parameter item information 117. The information processing system 100 may accumulate a parameter item setting history for each of a plurality of skills and include the parameter setting history 119 in the query. The information processing system 100 may include all parameters previously set for parameter items indicated by the required parameter item information 117 in the query. The information processing system 100 may include the parameter setting history 119 going back a predetermined period in the query. The content set for a parameter item may be unique to each user 202. Therefore, including the parameter setting history 119 in the query may make it easier for the generation AI 400 to generate parameters suitable for the user 202, thereby increasing the likelihood that the parameters presented to the user 202 will be adopted by the user 202 as is.

[0069] 11 shows an example of a functional configuration of the information processing system 100. The information processing system 100 includes a storage unit 110 and a UI unit .

[0070] The storage unit 110 stores various types of information. The storage unit 110 stores, for example, skill management information 112. The storage unit 110 stores, for example, characteristic information 118. The storage unit 110 stores, for example, required parameter item information 117. The storage unit 110 stores, for example, a dialogue history 114. The storage unit 110 stores, for example, a call history 115. The storage unit 110 stores, for example, a parameter setting history 119.

[0071] The UI unit 130 functions as an interface with the user 202. The UI unit 130 includes a presentation unit 132, a user input reception unit 134, a processing unit 136, a situation recognition unit 138, an operation-related information acquisition unit 140, a query generation unit 142, a skill candidate identification unit 144, a candidate parameter identification unit 146, a setting unit 148, and a learning execution unit 150. It is not essential that the UI unit 130 include all of these units.

[0072] The presentation unit 132 presents various information to the user 202, and the user input reception unit 134 receives various inputs from the user 202. The presentation unit 132 may display various information on the screen of the computer 200. The user input reception unit 134 may acquire a user input 111 input via an input device of the computer 200.

[0073] The processing unit 136 executes various processes. The processing unit 136 performs input and output with the user 202 via the presentation unit 132 and the user input reception unit 134. The processing unit 136 may execute various applications in accordance with instructions from the user 202 to perform input and output with the user 202. The processing unit 136 may execute various skills in accordance with instructions from the user 202 to perform input and output with the user 202.

[0074] The storage unit 110 adds the skills that the user 202 has called and used to the call history 115. The storage unit 110 adds the parameters that the user 202 has set in the parameter items of the skills to the parameter setting history 119.

[0075] The processing unit 136 may execute a dialogue with the user 202. The processing unit 136, for example, executes an AI application and executes a dialogue with the user 202 through the AI ​​application. The storage unit 110 adds the content of the dialogue with the user 202 to the dialogue history 114. The processing unit 136, for example, mediates between the generated AI 400 and the user 202, thereby realizing the dialogue between the generated AI 400 and the user 202. The storage unit 110 adds the content of the dialogue between the generated AI 400 and the user 202 to the dialogue history 114.

[0076] The situation recognition unit 138 recognizes the operation situation of the user 202. The situation recognition unit 138 recognizes, for example, applications open on the computer 200. The situation recognition unit 138 recognizes, for example, display information 116. For example, the situation recognition unit 138 recognizes text content from among the information displayed on the screen of the computer 200 as the display information 116. For example, the situation recognition unit 138 recognizes, for example, a portion of the information displayed on the screen of the computer 200 that is highly likely to be the focus of the user 202 as the display information 116. For example, the situation recognition unit 138 recognizes, among the information displayed on the screen of the computer 200, information selected by the user 202 as the display information 116. For example, when multiple windows are displayed on the screen of the computer 200, the situation recognition unit 138 recognizes, as the display information 116, information included in an active window, information included in a top-level window, or information included in a window where the cursor or the like is located.

[0077] The operation-related information acquisition unit 140 acquires operation-related information related to an operation performed by the user 202 on the computer 200 operated by the user 202. For example, the operation-related information acquisition unit 140 acquires the user input 111 acquired by the user input acceptance unit 134 as the operation-related information. For example, the operation-related information acquisition unit 140 acquires an application running on the computer 200 recognized by the situation recognition unit 138 as the operation-related information. For example, the operation-related information acquisition unit 140 acquires the display information 116 recognized by the situation recognition unit 138 as the operation-related information.

[0078] The query generation unit 142 generates a query to be input to the generation AI 400. The query generation unit 142 may generate the query based on the operation-related information acquired by the operation-related information acquisition unit 140 and the skill management information 112.

[0079] The skill candidate identification unit 144 identifies one or more skills as candidates from the multiple skills based on the query generated by the query generation unit 142. The presentation unit 132 may present the one or more skills identified by the skill candidate identification unit 144 to the user 202. The skill candidate identification unit 144 may input the query generated by the query generation unit 142 to the generation AI 400 and identify one or more skills output from the generation AI 400 as candidates.

[0080] The query generation unit 142 may generate a query for implementing the skill recommendation function (user-active version). For example, the query generation unit 142 generates a query including the user input 111 acquired by the user input receiving unit 134, the skill management information 112, and text indicating content to output one or more recommended skills from among multiple skills whose skill information is included in the skill management information 112 based on the user input 111. If there are multiple recommended skills, the query generation unit 142 may generate a query further including text indicating content to output the recommended order. The query generation unit 142 may generate a query further including at least one of the app information 113, the dialogue history 114, and the call history 115.

[0081] For example, the query generation unit 142 includes app information 113 in the query. When multiple applications are running on the computer 200, the query generation unit 142 may include app information 113 including multiple applications and the hierarchical relationship between the multiple applications in the query. For example, when multiple applications are running on the computer 200, the query generation unit 142 includes app information 113 including information only about the application located at the top level in the query. For example, when multiple applications are running on the computer 200, the query generation unit 142 includes app information 113 including information only about the application that is active in the query.

[0082] For example, the query generation unit 142 may include the dialogue history 114 in the query. The query generation unit 142 may include, in the query, the dialogue history 114 from a point in time that goes back a predetermined period from the time the query was generated. The query generation unit 142 may include, in the query, the dialogue history 114 from a point in time that goes back a predetermined period from the time the query was generated and dialogue tendency information generated from the dialogue history 114 from a point in time that goes back a predetermined period from the time the query was generated.

[0083] For example, the query generating unit 142 may include in the query the call history 115. The query generating unit 142 may include in the query the call history 115 including the order of use of skills that the user 202 has used in the past.

[0084] The query generation unit 142 may weight two or more of the application information 113, the interaction history 114, and the call history 115 when the query includes the two or more of the application information 113, the interaction history 114, and the call history 115. The query generation unit 142 may apply different weights to two or more of the application information 113, the interaction history 114, and the call history 115. For example, when the query generation unit 142 includes the application information 113, the interaction history 114, and the call history 115 in the query, the query generation unit 142 applies a first weight, a second weight heavier than the first weight, and a third weight heavier than the second weight. Which weight to apply to each of the application information 113, the interaction history 114, and the call history 115 may be set by an administrator of the information processing system 100 or the user 202. For example, the query generation unit 142 may include text indicating the weight applied to each of the application information 113, the interaction history 114, and the call history 115 in the query.

[0085] The skill candidate identification unit 144 may input the query generated by the query generation unit 142 into the generation AI 400 to obtain one or more skills output from the generation AI 400, and the presentation unit 132 may present the one or more skills to the user 202 as candidates for recommended skills in response to an active user input 111 by the user 202.

[0086] The query generation unit 142 may generate a query for realizing the skill recommendation function (user-passive version). For example, the query generation unit 142 generates a query including at least one of an application running on the computer 200 and the display information 116 acquired by the operation-related information acquisition unit 140, the skill management information 112, and text indicating content to output one or more recommended skills from among a plurality of skills whose skill information is included in the skill management information 112, based on at least one of the application running on the computer 200 and the display information 116. For example, the query generation unit 142 generates a query including an application running on the computer 200 acquired by the operation-related information acquisition unit 140, the skill management information 112, and text indicating content to output one or more recommended skills from among a plurality of skills whose skill information is included in the skill management information 112, based on the application running on the computer 200. For example, the query generation unit 142 generates a query including the display information 116 acquired by the operation-related information acquisition unit 140, the skill management information 112, and text indicating content to output one or more recommended skills from among a plurality of skills whose skill information is included in the skill management information 112, based on the display information 116. For example, the query generation unit 142 generates a query including the application running on the computer 200 and the display information 116 acquired by the operation-related information acquisition unit 140, the skill management information 112, and text indicating content to output one or more recommended skills from among a plurality of skills whose skill information is included in the skill management information 112, based on the application running on the computer 200 and the display information 116. The query generation unit 142 may generate a query that further includes text indicating content to output a recommendation order when there are a plurality of recommended skills.

[0087] The query generating unit 142 may include in the query the call history 115. The query generating unit 142 may include in the query the call history 115 including the order of use of skills that the user 202 has used in the past.

[0088] The query generation unit 142 may generate a query for realizing the parameter generation function. The query generation unit 142 may generate a parameter query for generating candidate parameters, which are candidates for parameters to be set in a target skill, based on required parameter item information 117 of the target skill for which parameters are to be generated, corresponding to a user input 111 by the user 202, and the operation-related information acquired by the operation-related information acquisition unit 140. For example, the query generation unit 142 generates a parameter query including the user input 111 acquired by the user input acceptance unit 134, the required parameter item information 117 of the target skill, and text indicating content for outputting recommended parameters for the parameter items indicated by the required parameter item information 117 based on the user input 111. The query generation unit 142 may generate a query that further includes text indicating content for outputting the order of recommendation when there are multiple recommended parameters.

[0089] The query generation unit 142 may generate a parameter query that further includes the dialogue history 114. The query generation unit 142 may include, in the parameter query, the dialogue history 114 from a predetermined period prior to the time the parameter query was generated. The query generation unit 142 may include, in the parameter query, the dialogue history 114 from a predetermined period prior to the time the parameter query was generated and dialogue tendency information generated from the dialogue history 114 from a predetermined period prior to the time the parameter query was generated.

[0090] The query generation unit 142 may generate a parameter query that further includes the characteristic information 118 of the target skill. The query generation unit 142 may generate a parameter query that further includes the parameter setting history 119 of the target skill.

[0091] The query generation unit 142 may weight multiple pieces of the dialogue history 114, the characteristic information 118, and the parameter setting history 119 when the query includes the multiple pieces of the dialogue history 114, the characteristic information 118, and the parameter setting history 119. The query generation unit 142 may apply different weights to multiple pieces of the dialogue history 114, the characteristic information 118, and the parameter setting history 119. For example, when the dialogue history 114, the characteristic information 118, and the parameter setting history 119 are included in the query, the query generation unit 142 applies a first weight, a second weight heavier than the first weight, and a third weight heavier than the second weight. Which weight to apply to each of the dialogue history 114, the characteristic information 118, and the parameter setting history 119 may be set by an administrator of the information processing system 100, the user 202, or the like. For example, the query generation unit 142 may include text indicating the weight applied to each of the dialogue history 114, the characteristic information 118, and the parameter setting history 119 in the query.

[0092] The candidate parameter identification unit 146 identifies candidate parameters based on the parameter query generated by the query generation unit 142. The candidate parameter identification unit 146 may input the parameter query to the generation AI 400 and identify parameters output from the generation AI 400 as candidate parameters.

[0093] The setting unit 148 sets the candidate parameters identified by the candidate parameter identification unit 146 to the target skill. The setting unit 148 may present the candidate parameters identified by the candidate parameter identification unit 146 to the user 202, and set the candidate parameters to the target skill in response to approval by the user 202.

[0094] The query generation unit 142 may generate a query such that the generation AI 400 outputs multiple parameters as candidates for each parameter item. For example, the query generation unit 142 generates a parameter query including the user input 111 acquired by the user input receiving unit 134, required parameter item information 117 for a target skill, and text indicating content to output multiple parameters recommended for each parameter item indicated by the required parameter item information 117 based on the user input 111. In this case, the setting unit 148 may present multiple parameters to the user 202 for each parameter item and set a parameter selected by the user 202 from the multiple parameters to the parameter item.

[0095] The learning execution unit 150 performs various types of learning. For example, the learning execution unit 150 performs learning regarding weighting of inputs to the generation AI 400. For example, when executing the skill recommendation function (user active version), the query generation unit 142 generates a query including a user input 111, skill management information 112, application information 113 to which a first weight, a second weight, and a third weight have been applied, an interaction history 114, and a call history 115. The skill candidate identification unit 144 inputs the query to the generation AI 400. The presentation unit 132 presents the multiple skills output from the generation AI 400 in the order of recommendation to the user 202. The processing unit 136 identifies the skill selected by the user 202 and stores the identified skill in the storage unit 110. The query generation unit 142 changes the weighting content each time it generates a query. For example, the query generation unit 142 changes the application destinations of the first weight, the second weight, and the third weight, or changes the magnitudes of the values ​​of the first weight, the second weight, and the third weight, each time a query is generated. The learning execution unit 150 executes machine learning using information accumulated in the storage unit 110, thereby identifying weight application details that result in a high probability that the skill ranked first in the recommendation order will be selected by the user 202. After the learning execution unit 150 identifies the weight application details, the query generation unit 142 may weight the app information 113, the dialogue history 114, and the call history 115 in accordance with the weight application details.

[0096] 12 shows an example of a process flow by the information processing system 100. Here, a process flow when the information processing system 100 executes the skill recommendation function (user active version) will be described.

[0097] In step (step may be abbreviated as S) 102, the user input accepting unit 134 acquires a user input 111 explicitly input by the user 202. The user input accepting unit 134 acquires, for example, the user input 111 input to an agent window.

[0098] In S104, the query generation unit 142 generates a query. The query generation unit 142 may generate a query including: a user input 111, skill management information 112, at least one of application information 113, dialogue history 114, and call history 115; text indicating content to output one or more recommended skills from among a plurality of skills whose skill information is included in the skill management information 112, based on the user input 111 and at least one of the application information 113, dialogue history 114, and call history 115; and text indicating content to also output a recommendation order when there are a plurality of recommended skills.

[0099] In S106, the skill candidate identification unit 144 inputs the query generated by the query generation unit 142 in S104 to the generation AI 400, and identifies one or more skills output from the generation AI 400 as candidate skills.

[0100] In S108, the presentation unit 132 presents the candidate skills identified by the skill candidate identification unit 144 in S106 to the user 202. The presentation unit 132 presents the candidate skills identified by the skill candidate identification unit 144 to the user 202 in a manner that prioritizes the candidate skills identified by the skill candidate identification unit 144 from among the multiple skills, for example. The manner of prioritization may be, for example, placing the candidate skills at the top of the order. The presentation unit 132 may present only the candidate skills identified by the skill candidate identification unit 144 from among the multiple skills to the user 202.

[0101] In response to a skill being selected by the user 202 (YES in S110), the process proceeds to S212, where the processing unit 136 executes the selected skill.

[0102] 13 schematically illustrates an example of the flow of processing by the information processing system 100. Here, the flow of processing when the information processing system 100 executes the skill recommendation function (user passive version) will be described.

[0103] In S202, the situation recognition unit 138 recognizes the operation situation of the user 202 on the computer 200. The situation recognition unit 138 may recognize at least one of an application running on the computer 200 and the display information 116. If there is a change in the operation situation of the user 202, the process proceeds to S204. For example, if the situation recognition unit 138 recognizes that the user 202 has made an input or that the user has made a selection, the process proceeds to S204.

[0104] In S204, the query generation unit 142 generates a query. The query generation unit 142 may generate a query including: at least one of an application running on the computer 200 and the display information 116 recognized in S202, the skill management information 112, and text indicating content to output one or more recommended skills from among a plurality of skills whose skill information is included in the skill management information 112, based on at least one of the application running on the computer 200 and the display information 116, and text indicating content to also output a recommendation order when there are a plurality of recommended skills.

[0105] In S206, the skill candidate identification unit 144 inputs the query generated by the query generation unit 142 in S204 to the generation AI 400, and identifies one or more skills output from the generation AI 400 as candidate skills.

[0106] In S208, the presentation unit 132 presents the candidate skills identified by the skill candidate identification unit 144 in S206 to the user 202. For example, the presentation unit 132 presents the candidate skills to the user 202 in a manner that prioritizes the candidate skills identified by the skill candidate identification unit 144 among the multiple skills. The presentation unit 132 may present only the candidate skills identified by the skill candidate identification unit 144 among the multiple skills to the user 202.

[0107] In response to a skill being selected by the user 202 (YES in S210), the process proceeds to S212, where the processing unit 136 executes the selected skill.

[0108] 14 shows an example of a process flow performed by the information processing system 100. Here, a process flow when the information processing system 100 executes the parameter generation function will be described.

[0109] In S302, the user input receiving unit 134 acquires the user input 111 explicitly input by the user 202. The user input receiving unit 134 acquires, for example, the user input 111 input to the agent window.

[0110] In S304, the query generation unit 142 generates a query. The query generation unit 142 may generate a query including: the user input 111, the skill management information 112, at least one of the application information 113, the dialogue history 114, and the call history 115; text indicating content to output one or more recommended skills from among a plurality of skills whose skill information is included in the skill management information 112, based on the user input 111 and at least one of the application information 113, the dialogue history 114, and the call history 115; and text indicating content to also output a recommendation order when there are a plurality of recommended skills.

[0111] In S306, the skill candidate identification unit 144 inputs the query generated by the query generation unit 142 in S304 to the generation AI 400, and identifies one or more skills output from the generation AI 400 as candidate skills.

[0112] In S308, the presentation unit 132 presents to the user 202 the candidate skills identified by the skill candidate identification unit 144 in S306. The presentation unit 132 presents to the user 202, for example, in a manner that prioritizes the candidate skills identified by the skill candidate identification unit 144 among the multiple skills. The manner of prioritization may be, for example, by placing the candidate skills at the top of the order. The presentation unit 132 may present to the user 202 only the candidate skills identified by the skill candidate identification unit 144 among the multiple skills. In response to a selection of any skill by the user 202 (YES in S310), the process proceeds to S312.

[0113] In S312, the query generation unit 142 generates a query for generating candidate parameters for the target skill selected in S310. The query generation unit 142 may generate a query including the user input 111 acquired by the user input receiving unit 134 in S302, and if there is a user input 111 for the skill selected in S310, the user input 111, the necessary parameter item information 117 for the target skill, text indicating content to output recommended parameters for the parameter items indicated by the necessary parameter item information 117 based on the user input 111, and, if there are multiple recommended parameters, text indicating content to also output the order of recommendation.

[0114] In S314, the candidate parameter identifying unit 146 inputs the query generated by the query generating unit 142 in S312 to the generating AI 400, and identifies the parameters output from the generating AI 400 as candidate parameters.

[0115] In S316, the presentation unit 132 presents the candidate parameters identified by the candidate parameter identification unit 146 in S314 to the user 202. For example, the presentation unit 132 inputs the candidate parameters in an unconfirmed state into the parameter item of the target skill and presents them to the user 202. When there are multiple candidate parameters for one parameter item, the presentation unit 132 may present the multiple candidate parameters as options for that parameter item.

[0116] In response to receiving an instruction to execute a skill using candidate parameters from the user 202 (YES in S318), the process proceeds to S320. In S320, the processing unit 136 executes the skill using the parameters for which the execution instruction has been received.

[0117] 15 schematically illustrates an example of the hardware configuration of a computer 1200 that functions as the information processing system 100. A program installed on the computer 1200 can cause the computer 1200 to function as one or more "parts" of an apparatus according to the present embodiment, or can cause the computer 1200 to perform operations associated with the apparatus according to the present embodiment or one or more "parts," and / or can cause the computer 1200 to perform a process according to the present embodiment or steps of the process. Such a program can be executed by the CPU 1212 to cause the computer 1200 to perform specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0118] The computer 1200 according to this embodiment includes a CPU 1212, a RAM 1214, and a graphics controller 1216, which are interconnected by a host controller 1210. The computer 1200 also includes input / output units such as a communications interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, or the like. The storage device 1224 may be a hard disk drive, a solid-state drive, or the like. The computer 1200 also includes a ROM 1230 and legacy input / output units such as a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0119] The CPU 1212 operates according to programs stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphics controller 1216 acquires image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into the graphics controller itself, and causes the image data to be displayed on the display device 1218.

[0120] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0121] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or programs that depend on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0122] The programs are provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, RAM 1214, or ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and causes cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by implementing operations or processing of information in accordance with the use of the computer 1200.

[0123] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded into the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0124] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and may perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0125] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and may undergo information processing. The CPU 1212 may perform various types of processing on data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries, each having an attribute value of a first attribute associated with an attribute value of a second attribute, are stored on the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0126] The above-described programs or software modules may be stored in a computer-readable storage medium on or near the computer 1200. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0127] The blocks in the flowcharts and block diagrams in the present embodiments may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, including integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as field programmable gate arrays (FPGAs) and programmable logic arrays (PLAs), including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0128] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by an appropriate device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture, including instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, etc. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), electrically erasable programmable read-only memory (EEPROM), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), Blu-ray disc, memory stick, integrated circuit card, etc.

[0129] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0130] Computer-readable instructions may be provided to a general-purpose computer, a special-purpose computer, or another programmable data processing device, or a programmable circuit, either locally or via a local area network (LAN), a wide area network (WAN) such as the Internet, so that the processor of the programmable data processing device, such as a computer, or the programmable circuit executes the computer-readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Here, the computer may be a personal computer (PC), a tablet computer, a smartphone, a workstation, a server computer, a general-purpose computer, a special-purpose computer, or the like, or may be a computer system in which multiple computers are connected. Such a computer system in which multiple computers are connected is also called a distributed computing system, and is a broad definition of computers. In a distributed computing system, multiple computers collectively execute a program by each executing a portion of the program and passing data between the computers as needed during program execution.

[0131] Examples of processors include computer processors, central processing units, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc. A computer may have one or more processors. In a multiprocessor system with multiple processors, each processor executes a portion of a program and passes data between processors as needed during program execution, allowing the multiple processors to collectively execute the program. For example, in multitasking, each of the multiple processors may execute a portion of each task in small chunks by switching tasks at time slice intervals. In this case, which portion of a program each processor executes changes dynamically. Which portion of a program each of the multiple processors executes may also be statically determined by multiprocessor-aware programming.

[0132] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0133] It should be noted that the execution order of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order. [Explanation of symbols]

[0134] 20 network, 100 information processing system, 110 memory unit, 111 user input, 112 skill management information, 113 application information, 114 dialogue history, 115 call history, 116 display information, 117 required parameter item information, 118 characteristic information, 119 parameter setting history, 121 recommended skills, 122 recommended parameters, 130 UI unit, 132 presentation unit, 134 user input reception unit, 136 processing unit, 138 situation recognition unit, 140 operation-related information acquisition unit, 142 query generation unit, 144 skill candidate identification unit, 146 candidate parameter identification unit, 148 setting unit, 150 learning execution unit, 200 computer, 202 user, 212 mail application window, 214 agent window, 222 calendar application window, 224 agent window, 230 agent window, 300 Server, 400, generation AI, 1200, computer, 1210, host controller, 1212, CPU, 1214, RAM, 1216, graphics controller, 1218, display device, 1220, input / output controller, 1222, communication interface, 1224, storage device, 1230, ROM, 1240, input / output chip

Claims

1. a storage unit that stores skill management information including skill information of a plurality of skills that constitute a workflow that links a plurality of types of services; an operation-related information acquisition unit that acquires operation-related information related to a user's operation on a computer operated by the user; a query generation unit that generates a query based on the operation-related information and the skill management information; a skill candidate identification unit that inputs the query to a generation AI and identifies one or more skills output from the generation AI as candidates; a presentation unit that presents the one or more skills identified by the skill candidate identification unit to the user; An information processing system comprising:

2. the operation-related information acquisition unit acquires a user input input by the user as the operation-related information; The information processing system of claim 1, wherein the query generation unit generates the query including at least one of an application running on the computer, an interaction history between the user and the generated AI, and a history of the user's use of the skill, the operation-related information, and the skill management information.

3. The information processing system according to claim 2 , wherein, when a plurality of applications are running on the computer, the query generation unit generates the query including the plurality of applications and a hierarchical relationship between the plurality of applications.

4. 3. The information processing system according to claim 2, wherein the query generation unit generates the query including the dialogue history, the dialogue history including a dialogue history with the user from a predetermined period going back from the time of generation of the query, and dialogue tendency information generated from a dialogue history with the user before the predetermined period going back from the time of generation of the query.

5. The information processing system according to claim 2 , wherein the query generation unit generates the query including the usage history including a usage ranking of the skills that the user has used in the past.

6. the operation-related information acquisition unit acquires, as the operation-related information, at least one of an application running on the computer and information displayed on a screen of the computer; 2. The information processing system according to claim 1, wherein the query generation unit generates the query including at least one of an application running on the computer and information displayed on a screen of the computer, and the skill management information.

7. The information processing system according to claim 6 , wherein the query generation unit generates the query further including a usage history of the skill by the user.

8. the operation-related information acquisition unit acquires a user input input by the user as the operation-related information; the query generation unit generates a parameter query for generating candidate parameters that are candidates for parameters to be set in the target skill, based on required parameter item information indicating parameter items required to execute the target skill and the operation-related information; The information processing system includes: a candidate parameter identification unit that identifies the candidate parameters based on the parameter query; a setting unit that sets the candidate parameters identified by the candidate parameter identification unit to the target skill; The information processing system according to claim 1 , comprising:

9. The information processing system according to claim 8, wherein the query generation unit generates the parameter query further including at least one of characteristic information indicating characteristics of the target skill, a dialogue history with the user, and a history of parameters previously set for the target skill.

10. The information processing system according to claim 1 , further comprising the generation AI.

11. 1. A computer-implemented information processing method, comprising: a storage step of storing, in a storage unit, skill management information including skill information of a plurality of skills constituting a workflow each linking a plurality of types of services; an operation-related information acquisition step of acquiring operation-related information related to a user's operation on a computer operated by the user; a query generating step of generating a query based on the operation-related information and the skill management information; an identifying step of identifying one or more candidate skills from the plurality of skills based on the query; a presentation step of presenting the one or more skills identified in the identification step to the user as candidates; An information processing method comprising:

12. A program for causing a computer to execute the information processing method according to claim 11.

Images