Information processing systems, information processing methods, and programs

Abstract

The system generates configuration information for AI support programs based on sales infrastructure data, and enables appropriate updates of the configuration information or sales infrastructure data based on the execution results of the AI ​​support programs. [Solution] The information processing system comprises: a base data acquisition unit that acquires base data including sales strategy information, relationship information between sales data, and sales work flow information; a setting information generation unit that generates setting information including the role of an AI support program, reference data, tools used, execution procedure, output format, and evaluation conditions based on the base data; an execution management unit that executes an AI support program based on the setting information and acquires execution logs and sales performance logs related to said execution; an update candidate generation unit that generates update candidates based on the execution logs and sales performance logs and classifies the update candidates into candidates to be reflected in the setting information and candidates to be reflected in the base data; and a reflection control unit that updates the setting information or regenerates the setting information after updating the base data according to the classification result.

Description

Technical Field

[0001] The present invention relates to an information processing system, an information processing method, and a program.

Background Art

[0002] A dynamic campaign optimization system has been proposed that quantifies the dependency on response results using a Bayesian network or MT system (Mahalanobis-Taguchi System) based on communication history information and consumer attribute information stored in a consumer database, extracts valid field patterns, and dynamically updates placement condition information (see Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of Invention

Problems to be Solved by the Invention

[0004] However, the technique described in Patent Document 1 described above is aimed at optimizing placement conditions in advertising campaigns, and does not consider the viewpoint of generating setting information for an AI (Artificial Intelligence) support program based on sales operation base data, and appropriately updating the setting information or the sales operation base data based on the execution result of the AI support program.

[0005] The present invention has been made in view of such a background, and an object of the present invention is to provide a technique capable of generating setting information for an AI support program based on sales operation base data, and appropriately updating the setting information or the sales operation base data based on the execution result of the AI support program.

Means for Solving the Problems

[0006] The main invention of the present invention for solving the above problems is an information processing system comprising: a base data acquisition unit that acquires base data including sales strategy information, relationship information between sales data, and sales work flow information; a setting information generation unit that generates setting information including the role of an AI support program, reference data, tools used, execution procedure, output format, and evaluation conditions based on the sales work base data; an execution management unit that executes the AI ​​support program based on the setting information and acquires execution logs and sales result logs related to said execution; an update candidate generation unit that generates update candidates based on the execution logs and sales result logs and classifies said update candidates into candidates to be reflected in the setting information and candidates to be reflected in the sales work base data; and a reflection control unit that updates the setting information or regenerates the setting information after updating the sales work base data according to the classification result.

[0007] Further issues and solutions disclosed in this application will be made clear in the section on embodiments of the invention and in the drawings. [Effects of the Invention]

[0008] According to the present invention, setting information for an AI support program can be generated based on sales business infrastructure data, and the setting information or sales business infrastructure data can be appropriately updated based on the execution results of the AI ​​support program. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example of the overall configuration of an information processing system. [Figure 2] This figure shows an example of the hardware configuration of management server 2. [Figure 3] This figure shows an example of the software configuration for management server 2. [Figure 4] This is a diagram illustrating the processing flow in an information processing system. [Modes for carrying out the invention]

[0010] <System Overview> The following describes an information processing system according to one embodiment of the present invention. The information processing system of this embodiment automatically generates setting information for an AI (Artificial Intelligence) support program based on sales business infrastructure data, generates update candidates based on logs obtained from the execution results of the AI ​​support program, classifies whether the update candidates should be reflected in the setting information or the sales business infrastructure data, and then continuously and appropriately improves the setting information for the AI ​​support program by differentially regenerating only the affected setting items. In this embodiment, the AI ​​support program includes an AI agent. An AI agent is a program that autonomously or semi-autonomously performs tasks such as acquiring reference data, calling external tools, inference, output generation, and requesting confirmation from users or administrators. The information processing system of this embodiment does not need to be a single physical server, but may be logically implemented by multiple devices on the cloud.

[0011] In this embodiment, sales infrastructure data refers to a collection of data that may include customer-specific sales strategy information, relationship information between sales-related entities, sales workflow information, reference data, sales performance information, and product or service constraint information. Customer-specific sales strategy information includes, for example, key measures, approach policies, and priorities for each customer. Relationship information between sales-related entities includes, for example, information showing the relationships between each entity such as customers, sales representatives, products, deals, and contracts. Sales workflow information includes, for example, processing procedures at each phase of a deal, roles of sales representatives, and approval routes. Reference data includes, for example, product catalogs, price lists, and past deal examples. Sales performance information includes, for example, deal closing rates, deal progress rates, and sales performance. Product or service constraint information includes, for example, available combinations, applicable conditions, and legal restrictions.

[0012] In this embodiment, the configuration information for the AI ​​support program is information that defines the operation of the AI ​​support program and may include roles, reference data, tools used, execution procedures, output formats, evaluation conditions, logging conditions, update conditions, and version information. The roles define the business functions that the AI ​​support program performs. The reference data defines the data that the AI ​​support program references when performing inference or generating output. The tools used define the external tools that the AI ​​support program can call. The execution procedures define the procedures that the AI ​​support program follows when performing processing. The output formats define the format of the output generated by the AI ​​support program. The evaluation conditions define the criteria for evaluating the output or operation of the AI ​​support program. The logging conditions define the type and granularity of logs to be recorded during the execution of the AI ​​support program. The update conditions define the conditions for updating the configuration information. The version information is information for managing the versions of the configuration information.

[0013] Figure 1 shows an example of the overall configuration of an information processing system. The information processing system in this embodiment includes a management server 2. The management server 2 is connected to the user terminal 1 via a communication network. The communication network is, for example, the internet and is constructed using public telephone lines, mobile phone lines, wireless communication channels, Ethernet (registered trademark), etc.

[0014] User terminal 1 is a computer operated by a sales representative or administrator. User terminal 1 can be, for example, a smartphone, tablet computer, or personal computer.

[0015] The management server 2 may be a general-purpose computer such as a workstation or personal computer, or it may be logically implemented through cloud computing.

[0016] <Management Server 2> Figure 2 shows an example of the hardware configuration of the management server 2. Note that the illustrated configuration is just one example, and other configurations are also possible. The management server 2 includes a CPU (Central Processing Unit) 201, memory 202, storage device 203, communication interface 204, input device 205, and output device 206. The storage device 203 stores various data and programs, such as a hard disk drive, solid-state drive, or flash memory. The communication interface 204 is an interface for connecting to a communication network, such as an adapter for connecting to Ethernet®, a modem for connecting to a public telephone network, a wireless communication device for wireless communication, or a USB (Universal Serial Bus) connector or RS232C connector for serial communication. The input device 205 is for inputting data, such as a keyboard, mouse, touch panel, button, or microphone. The output device 206 is for outputting data, such as a display, printer, or speaker. Furthermore, each functional unit of the management server 2, as described later, is realized by the CPU 201 reading programs stored in the storage device 203 into memory 202 and executing them, and each storage unit of the management server 2 is realized as part of the storage area provided by memory 202 and storage device 203.

[0017] Figure 3 shows an example of the software configuration of the management server 2. The management server 2 comprises a base data acquisition unit 211, a setting information generation unit 212, an execution management unit 213, an update candidate generation unit 214, an impact range identification unit 215, an reflection control unit 216, a verification execution unit 217, a reflection determination unit 218, an evaluation management unit 219, and a storage unit 231. <Management Server 2>

[0018] The following describes the functional components of the management server 2.

[0019] The storage unit 231 is a storage area provided by the storage device of the management server 2. The storage unit 231 stores sales business base data, AI (Artificial Intelligence) support program setting information, execution logs, processing logs, reference data logs, user operation logs, sales performance logs, update candidates, reflection results, and respective versions of these in association with each other. The storage unit 231 stores traceability information indicating the correspondence relationship between data elements in the sales business base data and each setting item of the setting information. The storage unit 231 stores, for each update candidate, an update target, update content, basis log, contribution degree to sales performance, cost-effectiveness, application condition, and reliability in association with each other. The storage unit 231 stores the version of the setting information and the version of the sales business base data at the time when the setting information of the relevant version is generated, in association with each other.

[0020] The base data acquisition unit 211 acquires sales business base data. The sales business base data may include sales strategy information for each customer, relationship information between sales-related entities, sales business flow information, reference data, sales performance information, and constraint information for products or services. The base data acquisition unit 211 can acquire the sales business base data from, for example, an external customer management system (CRM (Customer Relationship Management) system), a product management system, or an in-house database. The base data acquisition unit 211 stores the acquired sales business base data in the storage unit 231. When the sales business base data is updated, the base data acquisition unit 211 stores the updated data as a new version in the storage unit 231.

[0021] The setting information generation unit 212 generates setting information for the AI support program based on the sales business base data stored in the storage unit 231. The setting information may include the role of the AI support program, reference data, tools to be used, execution procedures, output format, evaluation conditions, log recording conditions, update conditions, and version information. The setting information generation unit 212 can, for example, generate the role and execution procedure of the AI support program based on customer-specific sales strategy information, set the processing order and branch conditions based on sales business flow information, and set the evaluation conditions based on the constraint information of products or services.

[0022] For each setting item of the generated setting information, the setting information generation unit 212 associates the data element in the sales business base data that is the generation source of the relevant setting item, and stores the associated data as traceability information in the storage unit 231. For example, when the item "prohibit simultaneous proposal of Product A and Product B" is generated as the evaluation condition of the setting information, the data element "Product A and Product B cannot be sold simultaneously" in the product constraint information in the sales business base data is associated as the generation source of the relevant setting item. This traceability information is referred to when the later-described influence range specifying unit 215 specifies the influence range of the update candidate.

[0023] The setting information generation unit 212 stores the generated setting information in the storage unit 231 as a new version. When the sales business base data is updated, the setting information generation unit 212 can regenerate the setting information based on the updated sales business base data. At this time, the setting information generation unit 212 can perform differential regeneration that regenerates only the affected setting items and maintains the unaffected setting items, based on the influence range specified by the later-described influence range specifying unit 215.

[0024] The execution management unit 213 executes the AI ​​support program based on the configuration information stored in the memory unit 231 and acquires various logs related to the execution. The AI ​​support program includes an AI agent. The AI ​​agent is a program that autonomously or semi-autonomously acquires reference data, calls external tools, performs inference, generates output, and requests confirmation from the user or administrator. When the AI ​​support program is executed, the execution management unit 213 records the configuration items used, reference data, tools used, and processing units in the sales business flow.

[0025] The logs acquired by the Execution Management Unit 213 include execution logs, processing logs, reference data logs, tool call logs, user operation logs, output modification logs, approval or rejection logs, and sales performance logs. The execution log records the startup, shutdown, and execution status of the AI ​​support program. The processing log records the content and results of each processing step executed by the AI ​​support program. The reference data log records the identification information and reference date and time of the data referenced by the AI ​​support program. The tool call log records the identification information, input parameters, and output results of external tools called by the AI ​​support program. The user operation log records the content of operations performed by the user during the execution of the AI ​​support program. The output modification log records the content of modifications made by the user to the output generated by the AI ​​support program. The approval or rejection log records the results of approval or rejection made by the user or administrator to proposals or outputs generated by the AI ​​support program. The sales performance log records sales results such as progress in negotiations, closing deals, and lost deals after the execution of the AI ​​support program. The execution management unit 213 stores each acquired log in the storage unit 231.

[0026] The execution management unit 213 stores each log in the storage unit 231, associating it with units of business negotiations, customers, configuration information, or business flows. For example, if an AI-assisted program generates a proposal during a business negotiation with a customer, the identification information of the negotiation, the version of the configuration information used, the version of the referenced product catalog, the modifications made by the user, and the final success or failure of the negotiation are recorded in association with each other.

[0027] The update candidate generation unit 214 generates update candidates based on each log stored in the storage unit 231 and classifies the update candidates. By analyzing each log, the update candidate generation unit 214 identifies areas in either the AI ​​support program configuration information or the sales business infrastructure data where improvement is deemed possible, and generates them as update candidates. The update candidate generation unit 214 classifies the generated update candidates to determine whether they should be updated in customer-specific sales strategy information, relationship information between sales-related entities, sales business flow information, reference data, or AI support program configuration information.

[0028] The classification of update candidates is performed, for example, as follows: In the output modification log, if a user repeatedly modifies a specific output format, it is classified as a candidate for updating the settings related to that output format. In the approval or rejection log, if a specific proposal is repeatedly rejected, it is classified as a candidate for updating the reference data or sales strategy information that underlies that proposal. In the sales results log, if the closing rate of deals following a specific business flow is low, it is classified as a candidate for updating the sales business flow information related to that business flow.

[0029] The update candidate generation unit 214 generates the update target, update content, justification log, contribution to sales results, cost-effectiveness, application conditions, and reliability for each update candidate, and stores them in the storage unit 231. The update target is the identification information of the data element or setting item to be updated. The update content is information indicating the value after the update or the direction of the change. The justification log is the identification information of the log that served as the basis for generating the update candidate. The contribution to sales results is an indicator showing the degree of improvement in sales results that can be expected if the update candidate is applied. Cost-effectiveness is an indicator showing the ratio of the cost required to apply the update candidate to the expected effect. Application conditions are information showing the conditions under which the update candidate should be applied. Reliability is an indicator showing the degree of confidence regarding the validity of the update candidate.

[0030] The impact scope identification unit 215 identifies data elements or setting items corresponding to the update candidate and, based on the traceability information stored in the storage unit 231, identifies the AI ​​support programs, setting items, reference data, business flows, or evaluation conditions that will be affected by the update candidate. The impact scope identification unit 215 stores the identified impact scope in the storage unit 231.

[0031] The impact scope identification unit 215 identifies the impact scope as follows, for example: If the update candidate indicates a change in product constraint information within the sales operations infrastructure data, the impact scope identification unit 215 refers to traceability information to identify the setting item that originates from the product constraint information. If the identified setting item is an evaluation condition, the execution procedure and output format associated with that evaluation condition are also identified as part of the impact scope. On the other hand, setting items that do not have a correspondence with the product constraint information are excluded from the impact scope. As a result, the reflection control unit 216, described later, can perform differential regeneration, regenerating only the affected setting items and maintaining the unaffected setting items.

[0032] The reflection control unit 216 updates or regenerates the setting information according to the classification result of the update candidate and the scope of impact identified by the scope of impact identification unit 215. If the update candidate is directly reflected in the setting information, the reflection control unit 216 updates the corresponding setting item and regenerates only the execution procedure or evaluation condition associated with that setting item. If the update candidate is reflected in the sales business infrastructure data, the reflection control unit 216 regenerates the setting information based on the sales business infrastructure data after the reflection. In this case, the reflection control unit 216 performs differential regeneration, regenerating only the affected setting items and maintaining the unaffected setting items, based on the scope of impact identified by the scope of impact identification unit 215.

[0033] When multiple update candidates affect the same data element or setting item, the reflection control unit 216 determines the order in which to apply or whether to accept the update candidates based on their contribution to sales results, cost-effectiveness, reliability, or approval status. For example, if multiple update candidates exist for the same setting item, the update candidate with the highest reliability exceeding a predetermined threshold and the greatest contribution to sales results is given priority for application.

[0034] The verification execution unit 217 generates the configuration information regenerated by the reflection control unit 216 as verification configuration information and performs a trial execution on past business negotiations, simulated business negotiations, or sample inputs. During the trial execution, the verification execution unit 217 verifies the regenerated configuration information using evaluation conditions corresponding to the affected configuration items. For example, the verification execution unit 217 takes past business negotiation data as input, performs a trial execution of the AI ​​support program based on the regenerated configuration information, and records its output. The verification execution unit 217 can also use a simulated business negotiation scenario to confirm whether the regenerated configuration information operates in accordance with the expected business flow.

[0035] The reflection determination unit 218 determines whether there is a violation of product constraints, use of prohibited data, deviation from the sales workflow, or a decline in performance, based on the results of the trial execution by the verification execution unit 217. Based on the determination result or administrator approval, the reflection determination unit 218 decides whether or not to reflect the regenerated setting information as production setting information. If the reflection determination unit 218 decides to reflect the setting information as production setting information, it stores the setting information in the storage unit 231 as production setting information and notifies the execution management unit 213 to use the production setting information for subsequent executions of the AI ​​support program. If the reflection determination unit 218 decides not to reflect the setting information as production setting information, it retains the setting information in the storage unit 231 as verification setting information and notifies the administrator of the determination result and the reasoning.

[0036] The evaluation management unit 219 stores each version of the sales operations infrastructure data, configuration information, logs, update candidates, and reflection results in the storage unit 231, associating them with each other. The evaluation management unit 219 compares sales results, negotiation progress rate, manual correction rate, processing time, token consumption, or constraint violation rate before and after the reflection of the production configuration information. If the evaluation results after the reflection do not meet predetermined conditions, the evaluation management unit 219 performs a rollback process to revert the corresponding sales operations infrastructure data or configuration information to a previous version. For example, if the negotiation progress rate falls below a predetermined threshold after the reflection of the production configuration information, the evaluation management unit 219 retrieves the configuration information of the version before the reflection from the storage unit 231 and notifies the execution management unit 213 to use that version of the configuration information.

[0037] The evaluation management unit 219 stores the version of the sales business infrastructure data, the version of the configuration information, the identification information of the update candidate, and the reflection results in the storage unit 231, associating them with each other. The evaluation management unit 219 reflects the evaluation results after the reflection in the classification, prioritization, or regeneration conditions of the next update candidate. For example, if sales results improve after applying a certain update candidate, the evaluation management unit 219 can notify the update candidate generation unit 214 to increase the reliability of similar update candidates.

[0038] Figure 4 is a diagram illustrating the processing flow in an information processing system.

[0039] The base data acquisition unit 211 acquires sales business base data, including customer-specific sales strategy information, relationship information between sales-related entities, sales business flow information, reference data, sales performance information, and product or service constraint information, and stores it in the storage unit 231 (S401). The setting information generation unit 212 generates setting information, including the role of the AI ​​support program, reference data, tools used, execution procedure, output format, evaluation conditions, log recording conditions, update conditions, and version information, based on the sales business base data stored in the storage unit 231, and stores it in the storage unit 231 along with traceability information that associates each setting item with the data element from which it was generated (S402). The execution management unit 213 executes the AI ​​support program based on the setting information stored in the storage unit 231, and acquires and stores in the storage unit 231 execution logs, processing logs, reference data logs, tool call logs, user operation logs, output modification logs, approval or rejection logs, and sales performance logs (S403). The update candidate generation unit 214 analyzes each log stored in the storage unit 231 to generate update candidates, and classifies the update candidates into which of the following should be updated: customer-specific sales strategy information, relationship information between sales-related entities, sales business flow information, reference data, or AI support program setting information, and stores them in the storage unit 231 (S404). The impact scope identification unit 215 identifies the data element or setting item corresponding to the update candidate and, based on traceability information, identifies the AI ​​support program, setting item, reference data, business flow, or evaluation condition that will be affected by the update candidate (S405). The reflection control unit 216 performs differential regeneration, regenerating only the affected setting items and maintaining the unaffected setting items, according to the classification result and impact scope of the update candidates (S406). The verification execution unit 217 uses the regenerated setting information as verification setting information and performs a trial execution on past sales negotiations, simulated sales negotiations, or sample inputs (S407). The reflection determination unit 218 determines, based on the results of the trial execution, whether there is a violation of product constraints, use of prohibited data, deviation from the sales business flow, or a decline in results, and decides whether or not to reflect the regenerated setting information as the production setting information based on the determination result or administrator approval (S408).The evaluation management unit 219 compares sales results, negotiation progress rate, manual correction rate, processing time, token consumption, or constraint violation rate before and after the application of the production configuration information. If the evaluation results after the application do not meet the predetermined conditions, it performs a rollback process to revert the corresponding sales business infrastructure data or configuration information to a previous version (S409).

[0040] As described above, the information processing system of this embodiment can automatically generate setting information for an AI support program based on sales infrastructure data, generate update candidates based on various logs obtained from the execution results of the AI ​​support program, classify whether the update candidates should be reflected in the setting information or the sales infrastructure data, identify the scope of impact based on the correspondence between data elements in the sales infrastructure data and setting items in the setting information, and differentially regenerate only the affected setting items. Furthermore, the regenerated setting information can be reflected as production setting information after verification and approval, and can be rolled back if the evaluation results after reflection do not meet predetermined conditions. This makes it possible to continuously and appropriately improve the setting information for the AI ​​support program and contribute to the efficiency of sales operations and the improvement of sales results.

[0041] Although these embodiments have been described above, they are intended to facilitate understanding of the present invention and are not intended to limit its interpretation. The present invention can be modified and improved without departing from its spirit, and equivalents thereof are also included.

[0042] For example, the processing performed by each functional unit of the management server 2 described above may be executed by any of the functional units. Furthermore, different functional units may be added to perform some of the processing performed by each of the functional units described above. Also, the functional units of the management server 2 may be distributed across multiple computers.

[0043] Furthermore, the information stored in each memory unit of the management server 2 may be stored in any of the memory units. That is, the information stored in the multiple memory units mentioned above may be stored in a single memory unit, or a portion of the information stored in one memory unit may be stored in another memory unit. <Example 1>

[0044] In the embodiment described above, an example was shown in which the update candidate generation unit 214 analyzes each log to generate an update candidate. However, the update candidate generation unit 214 may also integrate multiple update candidates to generate a single update candidate. For example, if the same setting item is repeatedly modified across multiple business opportunities in the output modification log, the update candidate generation unit 214 can integrate these multiple modification contents and generate a single update candidate that represents the modification trend. In this case, the reliability of the integrated update candidate is calculated based on the number and consistency of the individual modification logs. This reduces the impact of noise caused by individual logs and enables the generation of a more stable update candidate. <Modification 2>

[0045] In the embodiment described above, an example was shown in which the reflection determination unit 218 determines whether there is a violation of product constraints, use of prohibited data, deviation from the sales workflow, or decline in performance. However, the reflection determination unit 218 may also weight each determination item and decide whether or not to reflect it as production setting information based on the sum of the weighted determination scores. For example, a large weight may be set for product constraint violations and use of prohibited data, and if either one is detected, it may be decided not to reflect it. On the other hand, a small weight may be set for decline in performance, and if there are no problems with other determination items, reflection may be permitted subject to approval by the administrator. This makes it possible to flexibly adjust the determination criteria and realize reflection decisions according to business priorities. <Variation 3>

[0046] In the embodiment described above, an example was shown in which the evaluation management unit 219 performs a rollback process to revert the corresponding sales business infrastructure data or setting information to a previous version if the evaluation results after the update do not meet predetermined conditions. However, the evaluation management unit 219 may also determine whether the decline in the evaluation results after the update is temporary or continuous before performing the rollback process. Specifically, the evaluation management unit 219 sets a predetermined observation period after the update of the production setting information and monitors the trends in sales results, negotiation progress rate, and constraint violation rate during the observation period. If the decline in the evaluation results is observed only at the beginning of the observation period and shows an improving trend thereafter, the rollback process is postponed and the evaluation is performed again after the end of the observation period. On the other hand, if the decline in the evaluation results continues throughout the observation period, the rollback process is executed. This suppresses rollbacks due to temporary fluctuations and promotes stable improvement of the setting information. <Modification 4>

[0047] In the embodiment described above, an example was shown in which the setting information generation unit 212 generates setting information based on sales business infrastructure data. However, the setting information generation unit 212 may generate multiple different setting information candidates, compare these candidates based on the results of trial execution by the verification execution unit 217, and then adopt the candidate with the highest evaluation as the setting information. For example, the setting information generation unit 212 generates multiple candidates with different processing orders for the execution procedure, or multiple candidates with different combinations of reference data. The verification execution unit 217 performs trial execution using past sales negotiation data for each candidate and calculates the sales negotiation progress rate, manual correction rate, and constraint violation rate, respectively. The reflection determination unit 218 compares each of the calculated indicators and adopts the candidate with the highest overall evaluation as the setting information for production. This makes it possible to adopt setting information of higher quality compared to when a single setting information is generated. <Modification 5>

[0048] In the embodiment described above, an example was shown in which the impact scope identification unit 215 identifies the impact scope based on traceability information. However, the impact scope identification unit 215 may also identify the impact scope by referring to the application history of past update candidates in addition to traceability information. Specifically, the impact scope identification unit 215 obtains from the storage unit 231 the history of setting items that were actually affected when update candidates were applied to the same data element in the past, and compares this history with the identification result based on traceability information. Even if a setting item was not identified as part of the impact scope in the traceability information, if it was repeatedly affected in the past application history, that setting item is added to the impact scope. This makes it possible to identify the impact scope while considering potential dependencies that were not explicitly associated when the setting information was generated. <Variation 6>

[0049] In the embodiments described above, an example was shown in which the AI ​​support program includes an AI agent. However, the information processing system may also employ a multi-agent configuration in which multiple AI agents work in cooperation. In this case, the execution management unit 213 manages the role and execution order of each AI agent based on the configuration information and controls the process of passing the output of one AI agent as input to another AI agent. For example, an AI agent that collects customer information, an AI agent that generates proposal content based on the collected information, and an AI agent that evaluates the generated proposal content can each have independent configuration information and work in cooperation with each other. The update candidate generation unit 214 can generate update candidates for problems caused by cooperation between agents by individually analyzing the execution logs of each AI agent and analyzing logs related to data transfer between agents. The impact scope identification unit 215 identifies the impact that an update candidate on the configuration information of one AI agent has on the configuration information of other AI agents, and the reflection control unit 216 differentially regenerates the configuration items of all affected AI agents.

[0050] <Disclosure Items> Furthermore, this disclosure also includes the following configurations. [Item 1] A data acquisition unit acquires sales business infrastructure data, including sales strategy information, relationship information between sales data, and sales business flow information. Based on the aforementioned sales operations infrastructure data, a configuration information generation unit generates configuration information including the role of the AI ​​support program, reference data, tools used, execution procedures, output format, and evaluation conditions. An execution management unit that executes the AI ​​support program based on the aforementioned configuration information and acquires execution logs and sales performance logs related to said execution, An update candidate generation unit generates update candidates based on the execution log and the sales performance log, and classifies the update candidates into candidates to be reflected in the setting information and candidates to be reflected in the sales business infrastructure data. A reflection control unit that updates the setting information according to the classification result, or updates the sales business infrastructure data and then regenerates the setting information, An information processing system equipped with the following features. [Item 2] The setting information generation unit stores each setting item of the generated setting information in association with the data element in the sales business infrastructure data from which it was generated. The reflection control unit identifies the setting items affected by the update candidates based on the correspondence. The information processing system described in item 1. [Item 3] The execution management unit records the settings used, reference data, tools used, and processing units in the sales workflow when the AI-assisted program is executed. The information processing system described in item 1. [Item 4] The update candidate generation unit acquires the execution log and the sales performance log, associating them with units of business negotiations, customers, configuration information, or business flow. The information processing system described in item 1. [Item 5] The update candidate generation unit generates, for each update candidate, the update target, update content, justification log, contribution to sales results, cost-effectiveness, and application conditions, associating them with each other. The information processing system described in item 1. [Item 6] The reflection control unit, when the update candidate is reflected in the setting information, updates the corresponding setting item and regenerates the execution procedure or evaluation condition associated with that setting item. The information processing system described in item 1. [Item 7] The reflection control unit, when the update candidate is reflected in the sales business infrastructure data, regenerates the setting information based on the sales business infrastructure data after the reflection. The information processing system described in item 1. [Item 8] Steps include acquiring sales business infrastructure data, including sales strategy information, relationship information between sales data, and sales workflow information, Based on the aforementioned sales operations infrastructure data, the step of generating configuration information including the role of the AI ​​support program, reference data, tools used, execution procedure, output format, and evaluation conditions, The steps include: executing the AI ​​support program based on the aforementioned configuration information, and obtaining execution logs and sales performance logs related to the execution; The steps include generating update candidates based on the execution log and the sales performance log, and classifying the update candidates into candidates to be reflected in the setting information and candidates to be reflected in the sales business infrastructure data, The steps include updating the setting information according to the classification result, or updating the sales business infrastructure data and then regenerating the setting information, A method of information processing performed by a computer. [Item 9] Steps include acquiring sales business infrastructure data, including sales strategy information, relationship information between sales data, and sales workflow information, Based on the aforementioned sales operations infrastructure data, the step of generating configuration information including the role of the AI ​​support program, reference data, tools used, execution procedure, output format, and evaluation conditions, The steps include: executing the AI ​​support program based on the aforementioned configuration information, and obtaining execution logs and sales performance logs related to the execution; The steps include generating update candidates based on the execution log and the sales performance log, and classifying the update candidates into candidates to be reflected in the setting information and candidates to be reflected in the sales business infrastructure data, The steps include updating the setting information according to the classification result, or updating the sales business infrastructure data and then regenerating the setting information, A program that causes a computer to execute something. [Item 10] A data acquisition unit acquires sales business infrastructure data, including sales strategy information, relationship information between sales data, and sales business flow information. A setting information generation unit generates setting information including the role of the AI ​​support program, reference data, tools used, execution procedure, output format, and evaluation conditions based on the aforementioned sales business infrastructure data, and associates each setting item of the generated setting information with the data elements in the sales business infrastructure data from which it was generated. An execution management unit executes the AI ​​support program based on the aforementioned configuration information and acquires execution logs, processing logs, reference data logs, user operation logs, and sales performance logs related to the execution. An update candidate generation unit generates update candidates based on the aforementioned logs and classifies the update candidates into which of the following should be updated: the sales strategy information, the relationship information between sales data, the sales business flow information, the reference data, or the configuration information. An impact scope identification unit identifies the data element or setting item corresponding to the update candidate, and based on the correspondence, identifies the AI ​​support program, setting item, reference data, business flow, or evaluation condition affected by the update candidate. A reflection control unit that, depending on the classification result of the update candidates and the scope of impact, regenerates only the affected setting items, maintains the unaffected setting items, or regenerates the setting information after updating the sales business infrastructure data, An information processing system equipped with the following features. [Item 11] The update candidate generation unit generates, for each update candidate, the update target, update content, justification log, contribution to sales results, cost-effectiveness, application conditions, and reliability, associating them with each other. The information processing system described in item 10. [Item 12] The reflection control unit, when multiple update candidates affect the same data element or setting item, determines the order of application or acceptance / rejection of the update candidates based on their contribution to sales results, cost-effectiveness, reliability, or approval status. The information processing system described in item 10. [Item 13] The reflection control unit, when the update candidate is directly reflected in the setting information, updates the corresponding setting item and regenerates only the execution procedure or evaluation condition associated with that setting item. The information processing system described in item 10. [Item 14] The reflection control unit, when the update candidate is reflected in the sales business infrastructure data, regenerates the setting information based on the sales business infrastructure data after the reflection. The information processing system described in item 10. [Item 15] Steps include acquiring sales business infrastructure data, including sales strategy information, relationship information between sales data, and sales workflow information, Based on the aforementioned sales operations infrastructure data, the following steps are taken: generating configuration information including the role of the AI ​​support program, reference data, tools used, execution procedures, output format, and evaluation conditions; and associating each of the generated configuration items with the data elements in the source sales operations infrastructure data. The steps include executing the AI ​​support program based on the aforementioned configuration information and obtaining execution logs, processing logs, reference data logs, user operation logs, and sales performance logs related to the execution, The steps include generating update candidates based on the aforementioned logs, and classifying the update candidates to determine whether to update the sales strategy information, the relationship information between sales data, the sales workflow information, the reference data, or the configuration information; The steps include identifying the data element or setting item corresponding to the update candidate, and identifying the AI ​​support program, setting item, reference data, business flow, or evaluation conditions affected by the update candidate based on the correspondence, Depending on the classification results of the update candidates and the scope of impact, the steps include: regenerating only the affected setting items, maintaining the unaffected setting items, or regenerating the setting information after updating the sales business infrastructure data; A method of information processing performed by a computer. [Item 16] Steps include acquiring sales business infrastructure data, including sales strategy information, relationship information between sales data, and sales workflow information, Based on the aforementioned sales operations infrastructure data, the following steps are taken: generating configuration information including the role of the AI ​​support program, reference data, tools used, execution procedures, output format, and evaluation conditions; and associating each of the generated configuration items with the data elements in the source sales operations infrastructure data. The steps include executing the AI ​​support program based on the aforementioned configuration information and obtaining execution logs, processing logs, reference data logs, user operation logs, and sales performance logs related to the execution, The steps include generating update candidates based on the aforementioned logs, and classifying the update candidates to determine whether to update the sales strategy information, the relationship information between sales data, the sales workflow information, the reference data, or the configuration information; The steps include identifying the data element or setting item corresponding to the update candidate, and identifying the AI ​​support program, setting item, reference data, business flow, or evaluation conditions affected by the update candidate based on the correspondence, Depending on the classification results of the update candidates and the scope of impact, the steps include: regenerating only the affected setting items, maintaining the unaffected setting items, or regenerating the setting information after updating the sales business infrastructure data; A program that causes a computer to execute something. [Item 17] A data acquisition unit acquires sales business infrastructure data, including sales strategy information, relationship information between sales data, and sales business flow information. A setting information generation unit generates setting information including the role of the AI ​​support program, reference data, tools used, execution procedure, output format, and evaluation conditions based on the aforementioned sales business infrastructure data, and associates each setting item of the generated setting information with the data elements in the sales business infrastructure data from which it was generated. An execution management unit executes the AI ​​support program based on the aforementioned configuration information and acquires execution logs, processing logs, reference data logs, user operation logs, and sales performance logs related to the execution. An update candidate generation unit generates update candidates based on the aforementioned logs and classifies the update candidates into which of the following should be updated: the sales strategy information, the relationship information between sales data, the sales business flow information, the reference data, or the configuration information. An impact scope identification unit identifies the data element or setting item corresponding to the update candidate, and based on the correspondence, identifies the AI ​​support program, setting item, reference data, business flow, or evaluation condition affected by the update candidate. A verification execution unit generates configuration information that has been regenerated according to the classification results of the update candidates and the scope of impact, and executes a trial run on past business negotiations, simulated business negotiations, or sample inputs. Based on the results of the trial execution, a reflection determination unit determines whether there are any violations of product constraints, unauthorized use of data, deviations from the business flow, or decreased performance, and determines whether or not to reflect the regenerated configuration information as production configuration information based on the determination result or administrator approval. An evaluation management unit stores the versions of the sales operations infrastructure data, the configuration information, the logs, the update candidates, and the reflection results in association with each other, compares sales results, negotiation progress rate, manual correction rate, processing time, token consumption, or constraint violation rate before and after the reflection of the production configuration information, and if the evaluation results after the reflection do not meet predetermined conditions, reverts the corresponding sales operations infrastructure data or configuration information to a previous version. An information processing system equipped with the following features. [Item 18] The reflection determination unit, if the regenerated setting information is reflected as the production setting information, will use the production setting information for subsequent executions of the AI ​​support program. The information processing system described in item 17. [Item 19] The aforementioned evaluation management unit will reflect the evaluation results after the reflection in the classification, prioritization, or regeneration conditions for the next update candidates. The information processing system described in item 17. [Item 20] The verification execution unit verifies the regenerated setting information using evaluation conditions corresponding to the affected setting items during the trial execution. The information processing system described in item 17. [Item 21] The evaluation management unit stores the version of the sales business infrastructure data, the version of the setting information, the identification information of the update candidate, and the reflection result in a mutually corresponding manner. The information processing system described in item 17. [Item 22] Steps include acquiring sales business infrastructure data, including sales strategy information, relationship information between sales data, and sales workflow information, Based on the aforementioned sales operations infrastructure data, the following steps are taken: generating configuration information including the role of the AI ​​support program, reference data, tools used, execution procedures, output format, and evaluation conditions; and associating each of the generated configuration items with the data elements in the source sales operations infrastructure data. The steps include executing the AI ​​support program based on the aforementioned configuration information and obtaining execution logs, processing logs, reference data logs, user operation logs, and sales performance logs related to the execution, The steps include generating update candidates based on the aforementioned logs, and classifying the update candidates to determine whether to update the sales strategy information, the relationship information between sales data, the sales workflow information, the reference data, or the configuration information; The steps include identifying the data element or setting item corresponding to the update candidate, and identifying the AI ​​support program, setting item, reference data, business flow, or evaluation conditions affected by the update candidate based on the correspondence, The steps include generating configuration information regenerated according to the classification results of the update candidates and the scope of impact as verification configuration information, and performing a trial execution on past business negotiations, simulated business negotiations, or sample inputs, Based on the results of the trial execution, the steps include determining whether there are any violations of product constraints, unauthorized use of data, deviations from the business flow, or decreased performance, and deciding whether or not to reflect the regenerated configuration information as production configuration information based on the determination result or administrator approval, The steps include: storing the versions of the sales operations infrastructure data, the configuration information, the logs, the update candidates, and the reflection results in association with each other; comparing sales results, negotiation progress rate, manual correction rate, processing time, token consumption, or constraint violation rate before and after the reflection of the production configuration information; and, if the evaluation results after the reflection do not meet predetermined conditions, reverting the corresponding sales operations infrastructure data or configuration information to a previous version. A method of information processing performed by a computer. [Item 23] Steps include acquiring sales business infrastructure data, including sales strategy information, relationship information between sales data, and sales workflow information, Based on the aforementioned sales operations infrastructure data, the following steps are taken: generating configuration information including the role of the AI ​​support program, reference data, tools used, execution procedures, output format, and evaluation conditions; and associating each of the generated configuration items with the data elements in the source sales operations infrastructure data. The steps include executing the AI ​​support program based on the aforementioned configuration information and obtaining execution logs, processing logs, reference data logs, user operation logs, and sales performance logs related to the execution, The steps include generating update candidates based on the aforementioned logs, and classifying the update candidates to determine whether to update the sales strategy information, the relationship information between sales data, the sales workflow information, the reference data, or the configuration information; The steps include identifying the data element or setting item corresponding to the update candidate, and identifying the AI ​​support program, setting item, reference data, business flow, or evaluation conditions affected by the update candidate based on the correspondence, The steps include generating configuration information regenerated according to the classification results of the update candidates and the scope of impact as verification configuration information, and performing a trial execution on past business negotiations, simulated business negotiations, or sample inputs, Based on the results of the trial execution, the steps include determining whether there are any violations of product constraints, unauthorized use of data, deviations from the business flow, or decreased performance, and deciding whether or not to reflect the regenerated configuration information as production configuration information based on the determination result or administrator approval, The steps include: storing the versions of the sales operations infrastructure data, the configuration information, the logs, the update candidates, and the reflection results in association with each other; comparing sales results, negotiation progress rate, manual correction rate, processing time, token consumption, or constraint violation rate before and after the reflection of the production configuration information; and, if the evaluation results after the reflection do not meet predetermined conditions, reverting the corresponding sales operations infrastructure data or configuration information to a previous version. A program that causes a computer to execute something. [Explanation of Symbols]

[0051] 1 User terminal 2 Management Server

Claims

1. A base data acquisition unit that acquires and stores in a storage unit sales business base data including customer-specific sales strategy information, relationship information between sales-related entities, sales business flow information, reference data, and constraint information for products or services, A setting information generation unit generates the role and execution procedure of an AI support program based on the customer-specific sales strategy information, sets the processing order and branching conditions in the execution procedure based on the sales business flow information, and sets evaluation conditions based on the constraint information of the product or service, thereby generating setting information including the role, reference data, tools used, execution procedure, output format, and evaluation conditions, and stores traceability information in the storage unit that associates the identification information of each setting item of the generated setting information with the identification information of the data element in the sales business infrastructure data from which the setting item was generated. An execution management unit causes the AI ​​support program to perform the following in accordance with the execution procedure of the configuration information: acquiring the reference data, calling the external tool specified as the tool to be used, inference, and generating output according to the output format; acquiring an execution log including the configuration items used in the execution, identification information of the acquired reference data, identification information of the called external tool, input parameters to the external tool and output results, the content of the modifications made by the user to the generated output, and the result of approval or rejection of the output; and acquiring a sales performance log including the progress, closing, or loss of the deal after the execution; and storing the execution log and the sales performance log in the storage unit in association with the deal, customer, configuration information, or processing unit on the sales workflow. An update candidate generation unit analyzes the execution log and the sales performance log, identifies the setting items or data elements in the sales business infrastructure data to be updated based on the setting items, reference data or sales business flow identification information contained in the logs, generates update candidates, classifies the update candidates to be reflected in the setting information if the identified update target is a setting item in the setting information, and classifies the update candidates to be reflected in the sales business infrastructure data if the identified update target is a data element in the customer-specific sales strategy information, the relationship information between sales-related entities, the reference data or the sales business flow information, An impact scope identification unit identifies the corresponding data element or setting item based on the identification information of the update target included in the update candidate, and identifies the setting item associated with the identification information of the identified data element as a setting item affected by the update candidate by referring to the traceability information. If the update candidate is classified as a candidate to be reflected in the setting information, the corresponding setting item is updated, and only the execution procedure or evaluation condition associated with the identification information of the setting item is regenerated; if the update candidate is classified as a candidate to be reflected in the sales business infrastructure data, the corresponding data element is updated, and based on the updated sales business infrastructure data, only the setting items identified as affected are regenerated, while maintaining the setting items that were not identified as affected. An information processing system equipped with the following features.

2. The update candidate generation unit stores in the storage unit, for each update candidate, the update target which is the identification information of the setting item or data element to be updated, the update content which indicates the value after the update or the direction of the change, the basis log which is the identification information of the log that served as the basis for generating the update candidate, the degree of contribution to sales results, cost-effectiveness, application conditions, and reliability, in association with each update candidate. The information processing system according to claim 1.

3. The update candidate generation unit identifies the setting items related to the output format as the update target when the content of the correction included in the execution log is repeated for the same output format, identifies the reference data or the data elements of the customer-specific sales strategy information that formed the basis of the proposal as the update target when the rejection results included in the execution log are repeated for the same proposal, and identifies the data elements of the sales workflow information related to the sales workflow as the update target when the closing rate of a deal that went through a specific sales workflow in the sales results log is lower than a predetermined value. The information processing system according to claim 1.

4. When a plurality of update candidates affect the same data element or setting item, the reflection control unit determines the order in which to apply or whether to accept the update candidates based on their contribution to sales results, cost-effectiveness, reliability, or approval status. The information processing system according to claim 1.

5. A verification execution unit that uses the setting information regenerated by the reflection control unit as verification setting information and performs a trial execution on past business negotiations, simulated business negotiations, or sample inputs, The system further includes a reflection determination unit that determines whether there is a violation of product constraints, use of prohibited data, deviation from the sales workflow, or decline in performance based on the results of the trial execution, and determines whether or not to reflect the verification setting information as production setting information based on the determination result or administrator approval, If the execution management unit decides to reflect the settings as production configuration information, it will use the production configuration information for subsequent executions of the AI ​​support program. The information processing system according to claim 1.

6. The verification execution unit verifies the verification setting information using evaluation conditions corresponding to the setting items identified as affected setting items in the trial execution. The information processing system according to claim 5.

7. The evaluation management unit further comprises: storing the versions of the sales business infrastructure data, the setting information, the execution log, the sales results log, the update candidates, and the reflection results in association with each version in the storage unit; comparing sales results, negotiation progress rate, manual correction rate, processing time, token consumption, or constraint violation rate before and after the reflection of the production setting information; and, if the evaluation result after the reflection does not meet predetermined conditions, reverting the corresponding sales business infrastructure data or the setting information to a previous version. The information processing system according to claim 5.

8. The steps of acquiring and storing in a storage unit sales business infrastructure data including customer-specific sales strategy information, relationship information between sales-related entities, sales business flow information, reference data, and constraint information for products or services, The steps include generating the role and execution procedure of an AI support program based on the customer-specific sales strategy information, setting the processing order and branching conditions in the execution procedure based on the sales business flow information, and setting evaluation conditions based on the constraint information of the product or service, thereby generating setting information including the role, reference data, tools used, execution procedure, output format, and evaluation conditions, and storing traceability information in the storage unit that associates the identification information of each setting item in the generated setting information with the identification information of the data element in the sales business infrastructure data from which the setting item was generated, The steps include: causing the AI ​​support program to perform the acquisition of the reference data, the calling of the external tool specified as the tool to be used, inference, and output generation according to the output format, in accordance with the execution procedure of the setting information, acquiring an execution log including the setting items used in the execution, identification information of the acquired reference data, identification information of the called external tool, input parameters to the external tool and output results, the content of the modifications made by the user to the generated output, and the result of approval or rejection of the output, and acquiring a sales performance log including the progress, closing, or loss of the deal after the execution, and storing the execution log and the sales performance log in the storage unit in association with the deal, customer, setting information, or processing unit on the sales workflow; The steps include: analyzing the execution log and the sales results log, identifying the setting items or data elements in the sales business infrastructure data to be updated based on the setting items, reference data, or sales business flow identification information contained in the logs, generating update candidates; classifying the update candidates into candidates to be reflected in the setting information if the identified update target is a setting item in the setting information; and classifying the update candidates into candidates to be reflected in the sales business infrastructure data if the identified update target is a data element in the customer-specific sales strategy information, the relationship information between sales-related entities, the reference data, or the sales business flow information; The steps include: identifying the corresponding data element or setting item based on the identification information of the update target included in the update candidate; referring to the traceability information, identifying the setting item associated with the identification information of the identified data element as a setting item affected by the update candidate; If the update candidate is classified as a candidate to be reflected in the setting information, the corresponding setting item is updated, and only the execution procedure or evaluation condition associated with the identification information of the setting item is regenerated; if the update candidate is classified as a candidate to be reflected in the sales business infrastructure data, the corresponding data element is updated, and based on the updated sales business infrastructure data, only the setting items identified as affected setting items are regenerated, while maintaining the setting items that were not identified as such setting items; A method of information processing performed by a computer.

9. The steps of acquiring and storing in a storage unit sales business infrastructure data including customer-specific sales strategy information, relationship information between sales-related entities, sales business flow information, reference data, and constraint information for products or services, The steps include generating the role and execution procedure of an AI support program based on the customer-specific sales strategy information, setting the processing order and branching conditions in the execution procedure based on the sales business flow information, and setting evaluation conditions based on the constraint information of the product or service, thereby generating setting information including the role, reference data, tools used, execution procedure, output format, and evaluation conditions, and storing traceability information in the storage unit that associates the identification information of each setting item in the generated setting information with the identification information of the data element in the sales business infrastructure data from which the setting item was generated, The steps include: causing the AI ​​support program to perform the acquisition of the reference data, the calling of the external tool specified as the tool to be used, inference, and output generation according to the output format, in accordance with the execution procedure of the setting information, acquiring an execution log including the setting items used in the execution, identification information of the acquired reference data, identification information of the called external tool, input parameters to the external tool and output results, the content of the modifications made by the user to the generated output, and the result of approval or rejection of the output, and acquiring a sales performance log including the progress, closing, or loss of the deal after the execution, and storing the execution log and the sales performance log in the storage unit in association with the deal, customer, setting information, or processing unit on the sales workflow; The steps include: analyzing the execution log and the sales results log, identifying the setting items or data elements in the sales business infrastructure data to be updated based on the setting items, reference data, or sales business flow identification information contained in the logs, generating update candidates; classifying the update candidates into candidates to be reflected in the setting information if the identified update target is a setting item in the setting information; and classifying the update candidates into candidates to be reflected in the sales business infrastructure data if the identified update target is a data element in the customer-specific sales strategy information, the relationship information between sales-related entities, the reference data, or the sales business flow information; The steps include: identifying the corresponding data element or setting item based on the identification information of the update target included in the update candidate; referring to the traceability information, identifying the setting item associated with the identification information of the identified data element as a setting item affected by the update candidate; If the update candidate is classified as a candidate to be reflected in the setting information, the corresponding setting item is updated, and only the execution procedure or evaluation condition associated with the identification information of the setting item is regenerated; if the update candidate is classified as a candidate to be reflected in the sales business infrastructure data, the corresponding data element is updated, and based on the updated sales business infrastructure data, only the setting items identified as affected setting items are regenerated, while maintaining the setting items that were not identified as such setting items; A program that causes a computer to execute something.

Images