Information processing apparatus, allocation method, and computer-readable recording medium

Abstract

To appropriately allocate a plurality of generation models to a task (optimization of generation model to be allocated to task).SOLUTION: The information processing apparatus includes an evaluation result acquisition unit configured to acquire an evaluation result obtained by evaluating a plurality of generative models machine-learned to execute a given task to generate a deliverable, and an assignment unit configured to determine a plurality of generative models to be assigned to a target task to be executed, based on the evaluation result regarding the target task.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, an allocation method, and an allocation program. [Background technology]

[0002] Attempts are being made to apply AI (Artificial Intelligence) to various fields. For example, Patent Document 1 below describes an AI that communicates with users. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2019-8483 Summary of the Invention [Problem to be solved by the invention]

[0004] Patent Document 1 describes having a single AI (more specifically, a generative model such as a language model) execute the task of communicating with a user, but does not describe using multiple generative models in combination to execute a single task. If multiple generative models can be used in combination to execute a single task, it will be possible to execute a task that would be difficult to execute with a single generative model. In addition, various other effects can be expected, such as a reduction in the time required to complete task execution and an improvement in task execution accuracy.

[0005] However, when multiple generative models are allocated to a task to be executed, if the allocated generative models are inappropriate, not only will the above-mentioned effects not be expected, but the task execution time may be prolonged or the task execution accuracy may be reduced. Therefore, when multiple generative models are used in combination to execute a single task, a technology for appropriately allocating generative models is required, but such a technology is not known, and Patent Document 1 does not mention such a technology.

[0006] The present disclosure has been made in consideration of the above-mentioned problems, and one exemplary purpose thereof is to provide a technology that enables appropriate allocation of multiple generative models to tasks to be performed. [Means for solving the problem]

[0007] An information processing device according to an exemplary aspect of the present disclosure includes an evaluation result acquisition means for acquiring evaluation results obtained by evaluating a plurality of generative models that have been machine-learned to execute a given task and generate a deliverable, and an allocation means for determining a plurality of generative models to be allocated to a target task to be executed based on the evaluation results for the target task.

[0008] An allocation method according to an exemplary aspect of the present disclosure includes an evaluation result acquisition process in which at least one processor acquires evaluation results obtained by evaluating multiple generative models that have been machine-learned to execute a given task and generate a deliverable, and an allocation process in which, based on the evaluation results for a target task to be executed, multiple generative models are determined to be allocated to the target task.

[0009] An allocation program according to an exemplary aspect of the present disclosure causes a computer to function as an evaluation result acquisition means that acquires evaluation results obtained by evaluating multiple generative models that have been machine-learned to execute a given task and generate a deliverable, and an allocation means that determines multiple generative models to allocate to a target task based on the evaluation results for the target task to be executed. [Effects of the Invention]

[0010] According to one exemplary aspect of the present disclosure, an exemplary effect is achieved in that a technology is provided that enables appropriate allocation of multiple generative models to a task to be executed. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram illustrating a configuration of an information processing device according to the present disclosure. [Figure 2] FIG. 1 is a flow diagram illustrating the flow of an allocation method according to the present disclosure. [Figure 3] FIG. 10 is a block diagram showing a configuration of another information processing device according to the present disclosure. [Figure 4] FIG. 10 is a diagram illustrating an example of a UI (User Interface) screen that accepts the specification of a generative model. [Figure 5] FIG. 10 is a diagram illustrating an example in which the allocation of generative models is changed. [Figure 6] 4 is a flowchart showing the flow of processing executed by the information processing device shown in FIG. 3. [Figure 7] FIG. 1 is a block diagram illustrating a configuration of a computer that functions as an information processing device according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0012] The following are examples of embodiments of the present invention. However, the present invention is not limited to the exemplary embodiments shown below, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technologies (part or all of the products or methods) employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, embodiments obtained by appropriately omitting some of the technologies employed in the exemplary embodiments shown below may also be included in the scope of the present invention. Furthermore, the effects mentioned in the exemplary embodiments shown below are examples of effects expected in the exemplary embodiments, and do not define the scope of the present invention. In other words, embodiments that do not exhibit the effects mentioned in the exemplary embodiments shown below may also be included in the scope of the present invention.

[0013] First Exemplary Embodiment A first exemplary embodiment, which is an example of an embodiment of the present invention, will be described in detail with reference to the drawings. This exemplary embodiment is the basic form of each exemplary embodiment described later. The scope of application of each technology employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise. Furthermore, each technology shown in the drawings referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical obstacles arise.

[0014] (Configuration of information processing device 1) The configuration of an information processing device 1 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device 1. As shown in Fig. 1, the information processing device 1 includes an evaluation result acquisition unit 101 and an allocation unit 102.

[0015] The evaluation result acquisition unit 101 acquires evaluation results obtained by evaluating multiple generative models that have been machine-learned to execute a given task and generate a deliverable. The evaluation results may be acquired by any method. For example, the evaluation result acquisition unit 101 may acquire evaluation results by evaluating each generative model. Furthermore, for example, the evaluation result acquisition unit 101 may acquire evaluation results generated by the information processing device 1 or another device.

[0016] Here, the "generative model" may be any trained model that has been machine-learned to execute a given task and generate a deliverable, and the "task" and "deliverable" are arbitrary.

[0017] For example, if the task is to generate an answer to an input question, the deliverable is an answer generated by a generative model. To execute such a task, a language model that has been machine-learned to learn the arrangement of components (such as words) in a sentence or the arrangement of sentences in a document can be applied as the generative model. For example, the generative model can be a Generative Pre-Trained Transformer (GPT), which predicts a string that is likely to follow an input string and outputs a sentence containing the input string. Other examples of the generative model that can be used include a Text-to-Text Transfer Transformer (T5), a Bidirectional Encoder Representations from Transformers (BERT), a Robustly optimized BERT approach (RoBERTa), and an Efficiently Learning an Encoder that Classifies Token Replacements Accurately (ELECTRA).

[0018] For example, if the task is to generate a description of an input image, the deliverable is a description generated by a generative model. When executing such a task, a model that generates a description from an input image, generated by machine learning the relationship between the image and text indicating its content, may be applied as the generative model. For example, BLIP (Bootstrap Language Image Pre-Training) can be used as a generative model that generates a description of a still image, and Video-LLaVa can be used as a generative model that generates a description of a moving image.

[0019] For example, if the task is to generate an image according to the content of input text, the deliverable is an image generated by a generative model. When executing such a task, a model that generates an image according to the content of input text, generated by machine learning the relationship between the text and an image according to the content of the text, may be applied as the generative model.

[0020] Furthermore, the above-mentioned "evaluation" may be any evaluation result that can be used as a basis for determining a generative model to be assigned to a target task, which will be described later. For example, if the target task includes a step of generating an answer to a question, the evaluation result acquisition unit 101 may acquire an evaluation result that evaluates the accuracy of the answer to the question.

[0021] The above-mentioned "multiple generative models" also include generative models whose output tendencies are changed by prompt engineering. For example, consider a case where a general-purpose language model is applied as the above-mentioned generative model and this language model is made to execute a task of generating an answer to a question. In this case, the tendency of the generated answer will differ depending on whether or not the prompt input to the language model includes the sentence "Please answer in a way that is easy to understand even for a junior high school student." Therefore, when the above language model is used with this sentence included in the prompt, and when the above language model is used without this sentence included in the prompt, it can be considered that different generative models are used to execute the task.

[0022] Furthermore, by referencing specific data or a database when generating an output, one generative model 111A can function as multiple generative models 111A. The above "multiple generative models" also include generative models whose output trends are changed by referencing specific data or a database.

[0023] The allocating unit 102 determines multiple generative models to be allocated to a target task based on the evaluation results for the target task to be executed, among the evaluation results acquired by the evaluation result acquiring unit 101. Note that the target task may be a task that can be executed using all or part of the multiple evaluated generative models. For example, the target task may include multiple steps, each of which can be executed by one or multiple generative models. In this case, a deliverable for the entire target task is obtained by having the generative model generate a deliverable in each step.

[0024] As described above, the information processing device 1 according to this exemplary embodiment is configured to include an evaluation result acquisition unit 101 that acquires evaluation results obtained by evaluating multiple generative models that have been machine-learned to execute a given task and generate a deliverable, and an allocation unit 102 that determines multiple generative models to allocate to a target task based on the evaluation results for the target task to be executed.

[0025] Therefore, the information processing device 1 has the effect of being able to appropriately allocate multiple generative models to a task to be executed. For example, the information processing device 1 also makes it possible to optimize the generative models to be allocated to a target task.

[0026] (Allocation Program) The functions of the information processing device 1 described above can also be realized by a program. The allocation program according to this exemplary embodiment causes a computer to function as an evaluation result acquisition unit that acquires evaluation results of multiple generative models trained by machine learning to execute a given task and generate a deliverable, and as an allocation unit that determines multiple generative models to allocate to a target task based on the evaluation results for the target task. This allocation program provides the advantage of enabling multiple generative models to be appropriately allocated to the target task.

[0027] (Flow of allocation method) The flow of the allocation method according to this exemplary embodiment will be described with reference to Fig. 2. Fig. 2 is a flow diagram showing the flow of the allocation method. Note that the execution entity of each step in this allocation method may be a processor provided in the information processing device 1, or a processor provided in another device, or each step may be executed by a processor provided in a different device.

[0028] In S1 (evaluation result acquisition process), at least one processor acquires evaluation results of evaluating multiple generative models that have been machine-learned to execute a given task and generate a deliverable.

[0029] In S2 (allocation process), at least one processor determines a plurality of generative models to be allocated to a target task based on the evaluation result for the target task to be executed, among the evaluation results acquired in S1.

[0030] As described above, the allocation method according to this exemplary embodiment includes an evaluation result acquisition process in which at least one processor acquires evaluation results of multiple generative models trained by machine learning to execute a given task and generate a deliverable, and an allocation process in which, based on the evaluation results for the target task, multiple generative models are determined to be allocated to the target task. Therefore, the allocation method according to this embodiment has the advantage of being able to appropriately allocate multiple generative models to the target task.

[0031] Second Exemplary Embodiment A second exemplary embodiment, which is one example of an embodiment of the present invention, will be described in detail with reference to the drawings. Components having the same functions as those described in the above exemplary embodiment will be assigned the same reference numerals, and their description will be omitted as appropriate. The scope of application of each technology employed in this exemplary embodiment is not limited to this exemplary embodiment. That is, each technology employed in this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs. Furthermore, each technology shown in each drawing referenced to explain this exemplary embodiment can also be employed in other exemplary embodiments included in the present disclosure, to the extent that no particular technical hindrance occurs.

[0032] (Configuration of information processing device 1A) The configuration of an information processing device 1A according to this exemplary embodiment will be described with reference to FIG. 3. FIG. 3 is a block diagram showing the configuration of the information processing device 1A. The information processing device 1A is a device having a function of allocating a generative model to a target task to be executed. The information processing device 1A may be a device whose main function is to allocate a generative model, or may be a general-purpose device having other functions as well. Furthermore, the information processing device 1A may be a stationary device or a portable device.

[0033] As shown in the figure, information processing device 1A includes a control unit 10A that controls each unit of information processing device 1A and a storage unit 11A that stores various data used by information processing device 1A. Information processing device 1A also includes a communication unit 12A that enables information processing device 1A to communicate with other devices, an input unit 13A that accepts input to information processing device 1A, and an output unit 14A that enables information processing device 1A to output data. Control unit 10A includes an evaluation result acquisition unit 101A, an allocation unit 102A, a reception unit 103A, an evaluation unit 104A, an execution control unit 105A, and a presentation control unit 106A. Storage unit 11A stores a plurality of generative models 111A and evaluation results 112A.

[0034] The evaluation result acquisition unit 101A acquires evaluation results obtained by evaluating multiple generative models 111A that have been machine-learned to execute a given task and generate a deliverable, similar to the evaluation result acquisition unit 101 described in exemplary embodiment 1. Specifically, the evaluation result acquisition unit 101A acquires evaluation results 112A, which are data indicating the evaluation results of the multiple generative models 111A by the evaluation unit 104A, from the storage unit 11A.

[0035] The generative model 111A is a trained model that has been machine-learned to execute a given task and generate a deliverable, similar to the generative model described in the exemplary embodiment 1. A plurality of generative models 111A are stored in the storage unit 11A of the information processing device 1A shown in Fig. 3. Note that the generative model 111A does not necessarily need to be stored in the storage unit 11A, and it is also possible to use a generative model 111A external to the information processing device 1A.

[0036] The allocating unit 102A determines a plurality of generative models 111A to be allocated to a target task based on the evaluation result of the target task to be executed, among the evaluation results obtained by the evaluation result obtaining unit 101A, in the same manner as the allocating unit 102 described in the exemplary embodiment 1. A specific method of allocation will be described later.

[0037] The receiving unit 103A receives input from a user of the information processing device 1A. For example, the receiving unit 103A can also receive a designation of a generative model 111A to be allocated to a target task. Furthermore, for example, the receiving unit 103A can also receive an instruction to change the generative model 111A to be allocated to a target task.

[0038] The evaluation unit 104A evaluates the generative model 111A. The evaluation result of the generative model 111A by the evaluation unit 104A is stored in the storage unit 11A as an evaluation result 112A. The evaluation method applied by the evaluation unit 104A will be described in detail later.

[0039] The execution control unit 105A causes the generative model 111A to execute the target task in accordance with the allocation determined by the allocation unit 102A. The process of causing the generative model 111A to execute the task will be described in detail later.

[0040] The presentation control unit 106A presents various types of information to the user of the information processing device 1A. For example, the presentation control unit 106A can present the evaluation result 112A to the user. Furthermore, for example, the presentation control unit 106A can present the results generated by the generative model 111A in each of the multiple steps included in the target task to the user. The presentation method and manner are arbitrary. For example, the presentation control unit 106A can present the information by causing the output unit 14A to output the information, or can present the information by causing the communication unit 12A to output the information to a device other than the information processing device 1A. Furthermore, the presentation control unit 106A may present the information by displaying and outputting it, or by audio output, printout, or the like.

[0041] As described above, the information processing device 1A, like the information processing device 1 of exemplary embodiment 1, includes an evaluation result acquisition unit 101A that acquires evaluation results 112A obtained by evaluating multiple generative models 111A that have been machine-learned to execute a given task and generate a deliverable, and an allocation unit 102A that determines multiple generative models 111A to be allocated to a target task to be executed based on the evaluation results for the target task. Thus, an effect is obtained in which multiple generative models 111A can be appropriately allocated to the task to be executed.

[0042] (Use Case) As described in the first exemplary embodiment, the target task may be any task. For example, the target task may be to propose a countermeasure according to the patient's symptoms. In this case, the information processing device 1A assigns, to the target task, a generative model 111A that excels in eliciting the patient's symptoms and a generative model 111A that excels in estimating a countermeasure according to the symptoms. This makes it possible to propose an appropriate countermeasure according to the patient's symptoms. In this way, the information processing device 1A can also be applied to the medical field.

[0043] The information processing device 1A can also be applied to supporting the performance of work in an organization consisting of multiple members, such as a company. In this case, for example, by preparing a generative model 111A corresponding to each member, it is possible to have the generative model 111A corresponding to each member perform a target task, similar to when members form a team and perform a task. For example, it can also be used to create new ideas or solve problems between members.

[0044] It is also possible to consider the actual allocation of members by referring to the allocation by the information processing device 1A. For example, for a combination of generative models 111A that produced good results in executing a target task, the information processing device 1A may present to the user the combination of members that formed the basis of the generative model 111A as a combination suitable for executing the target task or a task similar to the target task. The evaluation result of the generative model 111A may also be used to understand the characteristics of the member that formed the basis of the generative model 111A. For example, when the information processing device 1A obtains an evaluation result indicating that a certain generative model 111A has high accuracy in answering questions in a specific technical field, it may output an estimation result indicating that the technical field is the specialty field of the member that formed the basis of the generative model 111A.

[0045] The information processing device 1A can also be applied to learning support in a community. In this case, a generative model 111A corresponding to each member of the community is prepared. This allows the generative model 111A to act on behalf of the members in discussions or to participate in the discussions. Furthermore, by conducting a discussion using a combination of generative models 111A corresponding to members with different areas of expertise, new knowledge can be generated from diverse perspectives. A target task can also be generating answers to questions or doubts posed in discussions between members. This allows appropriate answers to be generated by combining generative models 111A suitable for generating answers to such questions or doubts. Furthermore, by analyzing each generative model 111A and evaluating their learning tendencies, the knowledge learned by each generative model 111A can be visualized. This process can also be said to visualize the knowledge distribution of the members corresponding to each generative model 111A.

[0046] The information processing device 1A can also be applied to support creative activities and research and development. For example, generative models 111A corresponding to multiple creators, such as writers and manga artists, may be prepared. Then, multiple generative models 111A may be assigned to a target task, such as generating a new work or a part of it (e.g., story development, character setting, etc.). This allows for the creation of new works that would not have been created by a single creator. Furthermore, for example, by combining generative models 111A corresponding to creators or artists with different specialties, it becomes possible to generate novel works, or to use a generative model 111A corresponding to a deceased author to complete an unfinished work by that author.

[0047] The same applies when used to support research and development. For example, generative models 111A corresponding to various researchers in each field may be prepared. This makes it possible to combine generative models 111A of researchers in different fields or researchers with different specialties to create interdisciplinary research ideas, interpret experimental data, or construct hypotheses.

[0048] Furthermore, the information processing device 1A can also fine-tune a generative model 111A using training data used in the machine learning of another generative model 111A. For example, suppose a new researcher is assigned to a research team. In this case, the generative model 111A of the new researcher at the time of assignment has not learned the research know-how of the research team. Therefore, the generative model 111A of the new researcher is fine-tuned using training data used in the machine learning of the generative model 111A of a senior researcher in the research team. This makes it possible for the generative model 111A of the new researcher to generate a deliverable that takes into account the research know-how of the research team.

[0049] (Multiple generative models) The multiple generative models 111A to be assigned to a target task may be independently generated models. In this case, the model architecture of each generative model 111A may be the same or different. Furthermore, training data used for machine learning of each generative model 111A may be prepared independently. For example, by performing machine learning using a collection of statements by a well-known businessperson as training data, it is possible to generate a generative model 111A that generates comments similar to those of the well-known businessperson. Furthermore, the accuracy of the generated comments can be improved by including a variety of data sources, including documents and reports directly or indirectly related to the well-known businessperson, in the training data.

[0050] Furthermore, the generative model 111A may be generated by fine-tuning a base generative model using different training datasets. For example, a general-purpose language model may be fine-tuned using a training dataset that includes past comments by a specific person or documents created by that person. This allows the generation of a generative model 111A that generates comments similar to those of that person.

[0051] Furthermore, the generative model 111A may be generated by repeating the process of causing the generative model 111A to generate an output, providing feedback on the generated output, and updating the generative model 111A. For example, by providing feedback on whether a comment generated by the generative model 111A is appropriate as a comment from a specific person, it is possible to generate a generative model 111A that generates comments that are appropriate for that person.

[0052] Furthermore, as mentioned in the exemplary embodiment 1, prompt engineering can be used to cause one generative model 111A to function as multiple generative models 111A. Furthermore, as mentioned in the exemplary embodiment 1, by having predetermined data or a database referenced when generating an output, one generative model 111A can be used to function as multiple generative models 111A. For example, by having a certain generative model 111A reference a database that records the attributes, knowledge, experience, etc. of a specific person, it is possible to generate an output that reflects the attributes, knowledge, experience, etc. of that person. Therefore, by preparing such a database for each of multiple people and having the generative model 111A reference it, it is possible to generate an output corresponding to each person.

[0053] As described above, multiple generative models 111A can be generated using various methods, and one generative model 111A can function as multiple generative models 111A. These methods can also be combined. For example, a generative model 111A that has been fine-tuned and then further improved in accuracy through feedback can be used. It is also possible to generate a generative model 111A corresponding to a real person (a model that can generate output similar to that of the person). This makes it possible to build a system that promotes knowledge sharing and knowledge utilization within a real organization, for example, by preparing a generative model 111A corresponding to each member of the organization.

[0054] (Evaluation method) The evaluation method applied by the evaluation unit 104A will be described below. As described in the first exemplary embodiment, any evaluation method can be applied as long as it can provide evaluation results that can be used as a basis for determining the generative model 111A to be allocated to the target task.

[0055] For example, the structure of the generative model 111A (which can also be called the model architecture) significantly affects the performance of the generative model 111A. Therefore, the evaluation unit 104A may acquire information indicating the structure of the generative model 111A as the evaluation result or information for generating the evaluation result.

[0056] Examples of information indicating the structure of the generative model 111A include the algorithm, the number of model layers, the type of each layer such as a convolutional layer or a fully connected layer, the presence or absence of an attention mechanism, the number of parameters, and the values ​​of hyperparameters. The hyperparameters may also be parameters indicating, for example, a learning rate, a batch size, the number of epochs, the type of optimizer, etc.

[0057] The above information may be associated with each generative model 111A as metadata, or may be extracted from a predefined API (Application Programming Interface) or a serialized model file. The above-described hyperparameters may also be extracted from a training script or a configuration file. By previously associating various hyperparameters with the performance and characteristics of the generative model 111A, evaluation results of the performance and characteristics of the generative model 111A can be obtained from the values ​​of the various hyperparameters.

[0058] Furthermore, the training data used in the machine learning of the generative model 111A (including fine-tuning and feedback learning) determines the characteristics of the generative model 111A. For this reason, the evaluation unit 104A may acquire, as the evaluation result, information indicating an overview of the training data used in the machine learning of the generative model 111A (e.g., domain, data size, language, statistical information of each sample included in the training data, etc.). The domain may be, for example, medicine, law, IT (Information Technology), etc. The data size may be expressed, for example, by the number of samples or the number of tokens. Such information may also be extracted, for example, from a meta-information file accompanying the training dataset.

[0059] The evaluation unit 104A may also analyze the training data to evaluate the generative model 111A that is generated by machine learning the training data. For example, the evaluation unit 104A may analyze the properties of each sample included in the training data (e.g., the length of the sentence, the frequency of technical terms used, the diversity of the data, etc.) and evaluate the generative model 111A based on the analysis results.

[0060] The evaluation unit 104A may also evaluate the generative model 111A by analyzing past input / output data for the generative model 111A. For example, the evaluation unit 104A may analyze the length of the sentences, the frequency of technical terms used, the diversity of the data, and the like, for the data input to or output from the generative model 111A.

[0061] The evaluation unit 104A may also evaluate the content of past output responses (e.g., question answering, document generation, coding assistance, etc.), output accuracy (e.g., accuracy rate), and output speed (e.g., time required from start to completion of generation). Past input / output data can be collected, for example, from log files of API calls and interactions. If user feedback (e.g., satisfaction rating) has been provided for the output of the generative model 111A, the content of the feedback may also be taken into account in the evaluation.

[0062] The evaluation unit 104A can also estimate the tasks, fields of expertise, and level of expertise of the generative model 111A by analyzing the past input / output data and training data of the generative model 111A. For example, the evaluation unit 104A can estimate the tasks and fields of expertise of the generative model 111A by using natural language processing techniques such as topic modeling and keyword extraction.

[0063] The evaluation unit 104A may also cause each of the multiple generative models 111A to execute a standard task and evaluate each generative model 111A based on the execution results. For example, a task used in a known evaluation method may be applied as the standard task. Specifically, for the generative model 111A that inputs and outputs Japanese, JGLUE (Japanese General Language Understanding Evaluation), a method for evaluating the performance of natural language processing models, may be applied. For the generative model 111A that inputs and outputs English, GLUE (General Language Understanding Evaluation) may be applied. In these cases, the evaluation unit 104A evaluates the generative model 111A based on the execution results of tasks such as determining whether a hypothesis is "true," "false," or "neutral" for a sentence and a hypothesis pair, determining whether the sentence has a positive or negative content, determining whether multiple sentences have the same meaning, and identifying proper nouns in a sentence. Since these tasks have correct answers, the evaluation unit 104A compares the correct answers with the output of the generative model 111A, calculates the accuracy, precision, recall, F1 score, etc., and sets these as the evaluation results of the generative model 111A. The evaluation unit 104A may also measure the time required to complete the task. In this case, the measured time becomes the evaluation result of the answering speed.

[0064] The evaluation unit 104A may also evaluate the generative model 111A using an existing dataset such as the COCO (Common Objects in Context) dataset or the SQuAD (The Stanford Question Answering Dataset). For the generative model 111A with a translation function, the evaluation unit 104A can evaluate it using indices such as BLEU (Bilingual Evaluation Understudy), ROUGE (Recall-Oriented Understudy for Gisting Evaluation), or METEOR.

[0065] In addition, the evaluation unit 104A may cause the generative model 111A to execute, for example, a task of generating an answer to an input question, a task of generating a summary of an input document, and a task of generating program code in response to an input prompt. The evaluation criteria and evaluation method for the results of task execution may be determined in advance. For example, the criteria for a question answering task are the accuracy rate and naturalness of the answer, and for a document summarization task, the criteria are the accuracy and conciseness of the summary.

[0066] The evaluation unit 104A may also select multiple tasks (evaluation tasks) according to the target task to set a task set for evaluation. Alternatively, multiple types of task sets may be prepared in advance, and one that is suitable for the target task may be selected and used. In this case, it is preferable that each task set has a specific purpose or theme and reflects actual problems that the user may face.

[0067] Furthermore, it is preferable that the evaluation unit 104A evaluates the generative model 111A using an evaluation method according to the modality of input and output of the generative model 111A. For example, it is preferable that the evaluation unit 104A applies an evaluation method for evaluating the quality of an image to the generative model 111A that generates an image, and applies an evaluation method for evaluating the quality of audio to the generative model 111A that generates audio.

[0068] Note that the evaluation unit 104A may select a generative model 111A to execute a task before evaluating the generative model 111A by executing the task. For example, the evaluation unit 104A may select a generative model 111A based on information indicating the structure of each generative model 111A, or may select a generative model 111A according to the target task if the target task is determined. For example, if the target task includes a translation step, the evaluation unit 104A may select a generative model 111A that has a translation function. When there are a large number of available generative models 111A, it is preferable to perform such selection.

[0069] The evaluation unit 104A may also integrate multiple evaluation results to generate a final evaluation result. For example, the evaluation unit 104A may calculate an overall evaluation score by aggregating the evaluation scores obtained for each of multiple evaluation items. When aggregating the evaluation scores, normalization processing may be performed. This allows each generative model 111A to be evaluated using a consistent evaluation scale. For example, the evaluation unit 104A may also estimate the specialty or specialized knowledge area of ​​the generative model 111A from multiple evaluation results, and use the estimated result as the final evaluation result.

[0070] The evaluation unit 104A stores the above evaluation results as evaluation results 112A in the storage unit 11A. Then, the presentation control unit 106A presents the evaluation results 112A to the user. When presenting the evaluation results 112A to the user, the presentation control unit 106A may display the evaluation results 112A in a graph. The presentation control unit 106A may also present the evaluation results 112A using various data visualization tools. For example, by using a data visualization tool such as Grafana or Tableau, the evaluation results 112A can be presented in various formats such as a dashboard.

[0071] (Allocation based on evaluation results) The allocating unit 102A can automatically determine the generative model 111A to be allocated to the target task by using the evaluation result 112A generated as described above. For example, the allocating unit 102A may allocate to the target task, among the multiple generative models 111A for which evaluation results have been acquired, the generative model 111A whose evaluation result for the target task satisfies a predetermined condition. This provides the effect of automatically determining an appropriate generative model 111A to be allocated to the target task, in addition to the effect provided by the information processing device 1.

[0072] The "predetermined condition" can be set arbitrarily. For example, the evaluation result 112A indicates a score of 0 to 5 (the closer to 5, the higher the evaluation), which is an overall evaluation result obtained by aggregating evaluation results for multiple evaluation items. In this case, the "predetermined condition" may be set to selecting a predetermined number of generative models 111A whose scores are equal to or higher than a predetermined rank. Furthermore, for example, the "predetermined condition" may be set to selecting a predetermined number of generative models 111A whose scores are equal to or higher than a predetermined threshold.

[0073] Furthermore, the "predetermined condition" may be set for each of the multiple steps included in the target task. The "predetermined condition" may be set in advance, or the user may be allowed to set and change the "predetermined condition."

[0074] Furthermore, the allocation may take into consideration information other than the evaluation result 112A. For example, the allocation unit 102A may allocate generative models to target tasks based on attribute information indicating the attributes of each of the multiple generative models 111A. This provides the effect of enabling allocation that takes into consideration the attributes of the generative models 111A, in addition to the effect provided by the information processing device 1.

[0075] As attributes of the generative model 111A, various attributes can be applied, such as functional attributes such as having a translation function or an image generation function, attributes related to the input / output format such as limiting the input data format to text or allowing simultaneous input of both text and images, and attributes related to characteristics such as suitability for the legal field. Furthermore, if there is a person corresponding to the generative model 111A, the attributes of that person (for example, age (group), gender, occupation, etc.) may be considered as attributes of the corresponding generative model 111A.

[0076] The allocation method based on attributes is arbitrary. For example, a constraint on attributes may be set as the above-mentioned "predetermined condition." For example, the "predetermined condition" may include allocating only generative models 111A with predetermined attributes, or conversely, the "predetermined condition" may include allocating generative models 111A with different attributes to the target task.

[0077] The allocation unit 102A may also determine the allocation using a language model. In this case, the allocation unit 102A inputs the evaluation result 112A, the target task or its description, and attribute information of the candidate generative model 111A to be allocated to the target task into the language model along with a prompt instructing the language model to output the generative model 111A to be allocated to the target task taking into consideration the information. As a result, information indicating the generative model 111A to be allocated to the target task is output from the language model.

[0078] Furthermore, when a target task includes multiple steps, the allocation unit 102A may allocate a generative model 111A to each of the multiple steps based on the evaluation results for each of the multiple steps included in the target task. This provides the effect of allocating an appropriate generative model 111A to each step of the target task, in addition to the effect achieved by the information processing device 1. Note that the allocation unit 102A may allocate multiple generative models 111A to one step.

[0079] (Controlling execution of target tasks) As described above, the execution control unit 105A causes the generative model 111A to execute the target task according to the allocation determined by the allocation unit 102A. Specifically, the execution control unit 105A generates a prompt instructing the target task (or a step included in the target task) to be executed. The execution control unit 105A then inputs the generated prompt to the generative model 111A allocated to the target task (or a step included in the target task). As a result, a deliverable, which is the execution result of the target task, is output from the generative model 111A.

[0080] For example, suppose the target task is to generate a document in a predetermined format, and includes a first step of summarizing the content of an input article, a second step of generating a headline based on the content of the summary, and a third step of laying out the summary and headline to generate a document. In this case, the execution control unit 105A first inputs the article into the generative model 111A assigned to the first step to generate a summary. Next, the execution control unit 105A inputs the generated summary into the generative model 111A assigned to the second step to generate a headline based on the content of the article. The summary generated in the first step and the headline generated in the second step are intermediate deliverables of the target task. The execution control unit 105A then inputs the intermediate deliverables, the summary and headline, into the generative model 111A assigned to the third step to lay them out and generate a document, i.e., the final deliverable of the target task.

[0081] In this way, when a product of a previous step is used in a later step, it may not be possible to input the product of the previous step directly to the generative model 111A used in the later step. For example, this may occur when the generative model 111A used in the later step can only input text data, but the generative model 111A used in the previous step generates a product in a non-text format (e.g., an image). In such a case, the execution control unit 105A may convert the data format of the product of the previous step into a format that can be input to the generative model 111A of the later step. Alternatively, the allocation unit 102A may allocate the product in consideration of the input / output format of each generative model 111A and the order of use of each generative model 111A, so that the product of the previous step can be used in the later step without such conversion.

[0082] For example, suppose the target task is planning a new business and includes a first step of brainstorming ideas for the new business and a second step of generating presentation materials for the brainstormed ideas. In this case, the execution control unit 105A may input a prompt to each of the multiple generative models 111A assigned to the target task, requesting them to generate new business ideas, and output the new business ideas as deliverables. Next, the execution control unit 105A may input the multiple ideas that are deliverables of the first step into each of the multiple generative models 111A assigned to the target task, and generate presentation materials for each idea based on the other ideas. In this way, it is possible to execute the target task without assigning a generative model 111A to each step.

[0083] (UI screen example) As described above, the user may be prompted to specify the generative model 111A to be assigned to the target task. At this time, the presentation control unit 106A may display a UI screen for accepting the specification of the generative model 111A. This will be described with reference to FIG. 4. FIG. 4 is a diagram showing an example of the UI screen for accepting the specification of the generative model 111A. The UI screen shown in FIG. 4 includes a table a1 showing the evaluation results of the multiple generative models 111A, and relationship information a2 showing the relationships between the people who are the basis of the multiple generative models 111A.

[0084] Table a1 shows evaluation results for multiple evaluation items for multiple generative models 111A identified by identification information such as "AI001," "AI002," and "AI003." The presentation control unit 106A can generate table a1 using the evaluation results 112A generated by the evaluation unit 104A. Note that, hereinafter, the generative model 111A for "AI001" will be referred to as generative model AI001. The same applies to the other models.

[0085] The evaluation items shown in Figure 4 include research plan summary, business plan analysis, technical document classification, competitive analysis, project progress management, and customer feedback. These evaluation items are rated on a scale of 1 to 5, and the results are shown in Table a1. Table a1 also shows the average of these evaluation results as the overall score.

[0086] The research plan summary is a task to generate a summary of a research plan. The evaluation results for the evaluation items of the research plan summary shown in Table a1 are obtained by having the generative model 111A execute the task. For example, the evaluation criteria for this evaluation item include whether the main points can be accurately extracted from a complex plan, whether the summary is concise and easy to understand, etc. In this way, by having the model execute a task to summarize documents in a specific field, it is possible to determine whether the generative model 111A is suitable for that field.

[0087] The content analysis of a business plan involves analyzing the contents of the business plan and extracting key strategies, risks, and financial plans. The evaluation criteria for this assessment item include the ability to understand and analyze business plans. The classification of technical documents involves classifying technical documents. The evaluation criteria for this assessment item include the ability to accurately understand the technical content and appropriately classify them. Note that technical documents may be specific types of documents, such as patent documents. The competitive analysis is a task involving identifying competitors and comparing them with the company. The evaluation criteria for this assessment item include the ability to identify appropriate competitors and accurately identify the company's strengths and weaknesses relative to competitors. The project progress management is a task involving managing the project to ensure it progresses as planned. The evaluation criteria for this assessment item include the ability to grasp the project's progress and take appropriate measures in a timely manner to ensure the project progresses as planned. The customer feedback task involves analyzing the content of customer feedback. The evaluation criteria for this assessment item include the ability to accurately grasp the customer's intentions. In addition, various other tasks can be used to evaluate the generative model 111A, such as a task to evaluate a report summarizing the results of a competitive analysis, a task to predict market conditions, a task to create a report review, a task to create a new business proposal, a task to analyze a technical report, etc.

[0088] In the example of Fig. 4, the receiving unit 103A receives the specification of a generative model 111A via table a1. Specifically, in the example of Fig. 4, the receiving unit 103A receives the specification of two generative models 111A, namely, generative model AI001 and generative model AI003, via cursor Cur1. In table a1, the rows corresponding to these models are displayed so as to be distinguishable from the other rows, indicating that the specification of these models has been accepted. These generative models 111A specified by the user are assigned as the generative models 111A that execute the target task.

[0089] 4, the relationship information a2 indicates each person who is the basis of the multiple generative models 111A with an icon. The relationship information a2 is linked to table a1. Therefore, in the relationship information a2, the icon of the person corresponding to the generative model 111A selected in table a1 is displayed so as to be distinguishable from the icon of the person corresponding to the generative model 111A that is not selected.

[0090] The receiving unit 103A may receive the designation of the generative model 111A via the table a1, and may also receive the designation of the generative model 111A by the user via the relationship information a2. That is, the receiving unit 103A may receive an operation of designating a person (specifically, an icon in the example of FIG. 4) shown in the relationship information a2 as an operation of designating the generative model 111A corresponding to that person. This allows the user to easily designate a generative model 111A that takes into consideration the relationships between the people that form the basis of the multiple generative models 111A.

[0091] Furthermore, in the relationship information a2 shown in FIG. 4, the relationships between the people who formed the basis of the multiple generative models 111A are indicated by frames a21 and a22. More specifically, in the relationship information a2, icons corresponding to people who share common attributes are displayed within the same frame. For example, the icon of the person corresponding to generative model AI001 and the icon of the person corresponding to generative model AI002 are both displayed within frame a21. This indicates that these people share common attributes (for example, they belong to the same company). On the other hand, the icon of the person corresponding to generative model AI021 is displayed outside frame a21 and within frame a22. This indicates that this person has different attributes from the people corresponding to generative model AI001, generative model AI002, etc.

[0092] In this way, the presentation control unit 106A may present people with common attributes in association with each other. This allows the user to easily specify a generation model 111A that takes into account the attributes of each person on which the generation model 111A is based. For example, the user can easily specify multiple generation models 111A corresponding to people with common attributes, and can also easily specify multiple generation models 111A corresponding to people with different attributes.

[0093] Furthermore, in the relationship information a2 shown in FIG. 4, lines connecting each icon are displayed as information indicating the relationship between each person who is the basis of the multiple generative models 111A. For example, the line connecting the icon of a person corresponding to generative model AI001 and the icon of a person corresponding to generative model AI002 indicates that these people are in a superior-subordinate relationship. In this way, it is possible to display each person who is the basis of the multiple generative models 111A as a node and display the relationship between the people using edges connecting these nodes; in other words, it is possible to make the relationship information a2 a knowledge graph. Knowledge graphs make it possible to express any relationship other than a superior-subordinate relationship. The above nodes and edges can also be referred to as entities and relations, respectively.

[0094] Furthermore, when accepting the designation of a generative model 111A to be allocated to a target task, the presentation control unit 106A may present the results of analyzing past input / output data and training data for the generative model 111A that is a candidate for allocation. For example, the presentation control unit 106A may display, in a graph format, co-occurrence relationships identified by co-occurrence analysis of past input / output data and training data. Displaying such a graph can provide the user with information useful for determining allocation, such as the characteristics of the generative model 111A.

[0095] As described above, the information processing device 1A includes the presentation control unit 106A that presents the evaluation results to the user, and the reception unit 103A that receives the user's designation of the generative model 111A. The allocation unit 102A then allocates the user-designated generative model 111A to the target task. This provides the effect of realizing allocation that reflects the user's intention while taking into account the evaluation results of each generative model 111A, in addition to the effect provided by the information processing device 1.

[0096] Furthermore, as described above, each of the multiple generative models 111A may have a person who is the basis of the generative model 111A. In this case, the presentation control unit 106A may present to the user relationship information indicating the relationships between the people who are the basis of the multiple generative models 111A. This provides, in addition to the effects provided by the information processing device 1, an effect that allows the user to easily specify a generative model 111A that takes into consideration the relationships between the people who are the basis of the multiple generative models 111A.

[0097] (Allocation change) As described above, the receiving unit 103A may receive an instruction to change the allocation of generative models. This will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example in which the allocation of generative models has been changed.

[0098] Figure 5 shows two scenes, Scn1 and Scn2. Scn1 shows deliverables b1 to b3 generated by each generative model 111A when the target task is executed according to the initial allocation (i.e., the allocation before the change). Scn2 shows deliverables b1, b2', and b3' generated by each generative model 111A when the target task is executed according to the allocation after the change. The target task in this example is to generate ideas for a new business.

[0099] In the initial allocation shown in Scn1, the generative model AI001 is allocated to the first step of the target task, the generative model AI012 is allocated to the second step, and the generative model AI023 is allocated to the third step.

[0100] In Scn1, first, in the first step, according to the allocation described above, the generative model AI001 generates a comment as deliverable b1 that encourages the user to list the problems to be solved in the new business. For example, the execution control unit 105A may generate deliverable b1 by inputting to the generative model AI001 a description of the target task and each step included in the target task, and a prompt that instructs the user to generate a deliverable for the first step. Specifically, the execution control unit 105A may input the following prompt to the generative model AI001. In this case, a string that is the deliverable (for example, a string like the deliverable b1 shown in FIG. 5) is output in parentheses in the prompt "deliverable:{}."

[0101] Example prompt: Three people will work together to carry out the tasks listed below. The tasks will include steps 1 to 3 listed below. From step 2 onwards, deliverables will be generated using the deliverables from the preceding steps. You will generate deliverables in step 1 that will lead to good ideas in steps 2 and 3. Task: {Creating ideas for new businesses} Step: {First step, Second step:, Third step:} First step deliverable: {}" The execution control unit 105A, which has acquired the product of the first step in this way, causes the generative model AI012 assigned to the second step to generate the product of the second step. For example, the execution control unit 105A may input the following prompt to the generative model AI012. In this case, a character string representing the product (for example, a character string such as product b2 shown in FIG. 5) is output in parentheses in the prompt "product of the second step: {}".

[0102] Example prompt: Three people will work together to carry out the tasks listed below. The tasks will include steps 1 to 3 listed below. From step 2 onwards, deliverables will be generated using the deliverables from the preceding steps. You will use the deliverables from step 1 to generate deliverables for step 2 that will lead to good ideas in step 3. Task: {Creating ideas for new businesses} Step: {First step, Second step:, Third step:} Deliverables of the first step: {Let's come up with a new business idea. First, please list the challenges.} The second step's deliverable: {}" Then, the execution control unit 105A, which has acquired the product of the second step, causes the generative model AI023 assigned to the third step to generate the product of the third step. For example, the execution control unit 105A may input the following prompt to the generative model AI023. In this case, a character string representing the product (for example, a character string such as product b3 shown in FIG. 5) is output in parentheses in the prompt "product of the third step: {}".

[0103] Example prompt: Three people will work together to carry out the task below. The task will include steps 1 to 3 below. From step 2 onwards, you will generate a deliverable using the deliverables from the preceding steps. In step 3, you will generate a new business idea as a deliverable based on the deliverables from steps 1 and 2. Task: {Creating ideas for new businesses} Step: {First step, Second step:, Third step:} Deliverables of the first step: {Let's come up with a new business idea. First, please list the challenges.} The outcome of the second step: {In the mobility field, urban traffic problems would be an issue.} The third step's deliverable: {} The presentation control unit 106A may present the character string, which is the output of the third step generated as described above, to the user as the execution result of the target task. The presentation control unit 106A may also present to the user intermediate outputs until the execution result of the target task is obtained, i.e., the output of the first step and the output of the second step in the example of FIG. 5. In this case, the presentation control unit 106A may display icons representing the used generative models AI001, AI012, and AI023 in the order in which the generative models were used, as in the example of FIG. 5, and may also display outputs (character strings in this example) generated by each generative model in association with each icon. This allows the user to confirm the validity of the progress of the target task.

[0104] In this way, the information processing device 1A includes an execution control unit 105A that causes the generative model 111A assigned to each step to generate a deliverable in each of the multiple steps included in the target task, and that causes the execution control unit 105A to generate a deliverable in at least one of the second and subsequent steps based on the deliverable generated in the preceding step. This provides the effect of being able to generate a final deliverable using multiple deliverables generated by the multiple generative models 111A, in addition to the effect provided by the information processing device 1.

[0105] The presentation control unit 106A may present to the user the deliverables generated by the generative model 111A for each of the multiple steps included in the target task. The receiving unit 103A may also receive an instruction to change the generative model 111A allocated to each of the multiple steps. The execution control unit 105A may then generate a deliverable based on the deliverable generated in the preceding step for at least one of the second and subsequent steps in the changed allocation. This provides, in addition to the effects of the information processing device 1, the effect of being able to receive an instruction to change the allocation based on the presented deliverable and generate a new deliverable that reflects the change instruction.

[0106] 5, the receiving unit 103A receives an operation to move the icon of the generative model AI012 behind the icon of the generative model AI023. This operation is an operation to change the execution order of the processes of the generative models AI012 and AI023, that is, an operation to instruct switching of the generative models 111A allocated to the second and third steps.

[0107] When such an operation is accepted, the execution control unit 105A applies the changed allocation and re-executes the target task. In Scn2 after the re-execution, the deliverables in some steps are different from those in Scn1 due to the changed allocation. Specifically, the deliverables in the second and third steps have changed to b2' and b3', respectively. The deliverable in the second step has changed to b2' because the generative model 111A used in the second step has changed to generative model AI023. The deliverable in the third step has changed to b3' because the generative model 111A used in the third step has changed to generative model AI012 and the deliverable in the second step preceding the third step has changed to b2'. As such, in the example of FIG. 5, the deliverables in each step can be changed by a simple and intuitive operation of changing the position of the icon, allowing the user to easily generate the desired deliverable.

[0108] The receiving unit 103A may receive an instruction to replace the generative model 111A that was used before the allocation change with a generative model 111A that was not used before the allocation change. In this case, it is preferable that the presentation control unit 106A presents a list of specifiable generative models 111A to the user. The receiving unit 103A may also receive an instruction to add a generative model 111A to be allocated to the target task or an instruction to delete some of the generative models 111A allocated to the target task.

[0109] (Processing flow) The flow of processing executed by the information processing device 1A will be described with reference to Fig. 6. Fig. 6 is a flow diagram showing the flow of processing executed by the information processing device 1A. The flow of Fig. 6 includes each step of the allocation method according to this exemplary embodiment.

[0110] In S11, the receiving unit 103A receives the designation of a target task. The target task may be freely input in text format, for example, or candidate target tasks may be presented to the user and the user may select a target task from the candidates.

[0111] In S12 (evaluation result acquisition process), the evaluation result acquisition unit 101A acquires the evaluation result 112A stored in the storage unit 11A. As described above, the evaluation result 112A is an evaluation result obtained by evaluating a plurality of generative models 111A that have been machine-trained to execute a given task and generate a deliverable.

[0112] In S12, the evaluation unit 104A may evaluate each generative model 111A. In this case, the evaluation unit 104A functions as an evaluation result acquisition means. When evaluating the generative models 111A in S12, it is preferable that the evaluation unit 104A evaluates each generative model 111A by applying an evaluation method corresponding to the target task received in S11. The evaluation method to be applied to each target task may be determined in advance, or the evaluation method to be applied may be determined using a language model, etc. In the former case, for example, if the target task includes a step of generating an answer to a question, it may be determined that an evaluation method for evaluating the accuracy of the answer to the question is to be applied. In the latter case, for example, the target task, each applicable evaluation method, and a description of each evaluation method may be input into a language model, and the evaluation method to be applied to evaluate the target task may be output. Alternatively, the user may be allowed to specify the evaluation method to be applied.

[0113] In S13, the presentation control unit 106A presents to the user each of the generative models 111A that are candidates for allocation to the target task, and the evaluation results for those generative models 111A obtained in S12, along with relationship information indicating the relationships between the people on which those generative models 111A are based. For example, the presentation control unit 106A may present to the user each of the generative models 111A, their evaluation results, and relationship information by displaying an image such as that shown in FIG. 4. Note that presentation of the relationship information is not essential.

[0114] In S14, the receiving unit 103A receives a designation of the generative model 111A from among the generative models 111A presented in S13. For example, the user may designate the generative model 111A via the input unit 13A, or may designate the generative model 111A from another device via the communication unit 12A.

[0115] In S15 (allocation process), the allocation unit 102A determines a plurality of generative models 111A to be allocated to the target task based on the evaluation result of S12 for the target task to be executed. Specifically, the allocation unit 102A determines the generative model 111A designated by the user in consideration of the evaluation result of S12 presented in S13 as the generative model 111A to be allocated to the target task.

[0116] In S16, the execution control unit 105A causes the generative model 111A to execute the target task and generate a deliverable in accordance with the allocation determined in S15. Here, if the target task includes multiple steps, as described above, the execution control unit 105A may cause the second and subsequent steps to generate deliverables based on the deliverables generated in the preceding steps.

[0117] In S17, the presentation control unit 106A presents to the user the deliverables generated in S16. For example, as in the example of FIG. 5, the presentation control unit 106A may present deliverables generated in intermediate steps in addition to the final deliverable for the target task.

[0118] In S18, the receiving unit 103A determines whether or not there is an instruction to change the allocation. If the determination in S18 is YES, the process proceeds to S19, and if the determination in S18 is NO, the process in FIG. 6 ends. Note that if the determination in S18 is NO, the process may return to S11 and accept the designation of a new target task. The type of instruction to change the allocation can be set arbitrarily. For example, as in the example of FIG. 5, the receiving unit 103A may accept, as an instruction to change the allocation, an operation to move the position of a person icon corresponding to the generative model 111A whose allocation is to be changed to a position corresponding to the process after the change.

[0119] In S19, the executive control unit 105A applies the changed allocation. After this, the process returns to S16, and the executive control unit 105A causes each generative model 111A to generate a product in accordance with the changed allocation.

[0120] Note that the processes of S13 and S14 may be omitted. In this case, in S15, the allocation unit 102A automatically allocates to the target task, among the multiple generative models 111A whose evaluation results for the target task satisfy predetermined conditions, the generative models 111A whose evaluation results for the target task satisfy predetermined conditions. Also, some of the generative models 111A to be allocated to the target task may be designated by the user, and the other parts may be automatically determined by the allocation unit 102A.

[0121] [Modification] The execution entity of each process described in the above exemplary embodiment is arbitrary and is not limited to the above example. For example, a system having the same functions as the information processing devices 1 and 1A can be constructed using multiple devices that can communicate with each other. Furthermore, the execution entity of each process shown in the flowchart of FIG. 6 may be a single device (which can also be called a processor) or multiple devices (which can also be called processors).

[0122] [Software implementation example] Some or all of the functions of the information processing device 1, 1A may be realized by hardware such as an integrated circuit (IC chip), or may be realized by software.

[0123] In the latter case, the information processing devices 1 and 1A are realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in Fig. 7. Fig. 7 is a block diagram showing the hardware configuration of computer C that functions as information processing device 1 or 1A.

[0124] The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for causing the computer C to operate as the information processing device 1 or 1A. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the information processing device 1 or 1A.

[0125] The processor C1 may be, for example, a central processing unit (CPU), a graphic processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a tensor processing unit (TPU), a quantum processor, a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0126] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, mouse, display, and printer.

[0127] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0128] Furthermore, each of the above functions of the information processing devices 1 and 1A may be realized by a single processor provided in a single computer, by multiple processors provided in a single computer working together, or by multiple processors provided in each of multiple computers working together. Furthermore, a program for causing the information processing device 1 or 1A to realize each of the above functions may be stored in a single memory provided in a single computer, or may be distributed and stored in multiple memories provided in a single computer, or may be distributed and stored in multiple memories provided in each of multiple computers.

[0129] [Appendix 1] This disclosure includes the techniques described in the following appendices. However, the present invention is not limited to the techniques described in the following appendices, and various modifications are possible within the scope of the claims.

[0130] (Appendix A1) An information processing device comprising: an evaluation result acquisition means for acquiring evaluation results obtained by evaluating multiple generative models that have been machine-learned to execute a given task and generate a deliverable; and an allocation means for determining multiple generative models to be allocated to a target task to be executed based on the evaluation results for the target task.

[0131] (Appendix A2) The information processing device according to Appendix A1, wherein the allocation means allocates to the target task a generative model, of the plurality of generative models for which the evaluation results have been acquired, whose evaluation results for the target task satisfy a predetermined condition.

[0132] (Appendix A3) The information processing device according to appendix A1 or A2, wherein the allocation means allocates the generative models to the target task based on attribute information indicating attributes of each of the plurality of generative models.

[0133] (Appendix A4) The information processing device according to any one of appendices A1 to A3, wherein the allocation means allocates a generative model to each of a plurality of steps included in the target task based on an evaluation result for each of the plurality of steps.

[0134] (Appendix A5) An information processing device as described in Appendix A1, comprising: a presentation control means for presenting the evaluation results to a user; and a reception means for receiving a generative model specified by the user, wherein the allocation means allocates the generative model specified by the user to the target task.

[0135] (Appendix A6) An information processing device as described in Appendix A5, wherein each of the plurality of generative models has a person who is the basis of the generative model, and the presentation control means presents to the user relationship information indicating the relationship between each of the people who are the basis of the plurality of generative models.

[0136] (Appendix A7) An information processing device according to any one of appendices A1 to A6, comprising an execution control means for causing the generative model assigned to each step to generate a deliverable in each of a plurality of steps included in the task, and for at least one of the second and subsequent steps, causing the deliverable to be generated based on the deliverable generated in the preceding step.

[0137] (Appendix A8) An information processing device as described in Appendix A7, comprising: a presentation control means for presenting to a user a result generated by the generative model in each of the plurality of processes; and a receiving means for receiving an instruction to change the generative model to be allocated to each of the plurality of processes, wherein the execution control means generates a result in at least one of the second or subsequent processes in the allocation after the change based on a result generated in a preceding process.

[0138] (Appendix B1) An allocation method including: an evaluation result acquisition process in which at least one processor acquires evaluation results obtained by evaluating multiple generative models that have been machine-learned to execute a given task and generate a deliverable; and an allocation process in which, based on the evaluation results for a target task to be executed, multiple generative models are determined to be allocated to the target task.

[0139] (Appendix B2) The allocation method described in Appendix B1, wherein in the allocation process, the at least one processor allocates to the target task a generative model, of the plurality of generative models for which the evaluation results have been obtained, whose evaluation results for the target task satisfy predetermined conditions.

[0140] (Appendix B3) The allocation method according to appendix B1 or B2, wherein in the allocation process, the at least one processor allocates generative models to the target task based on attribute information indicating attributes of each of the plurality of generative models.

[0141] (Appendix B4) An allocation method according to any one of appendices B1 to B3, wherein in the allocation process, the at least one processor allocates a generative model to each of a plurality of steps included in the target task based on an evaluation result for each of the plurality of steps.

[0142] (Appendix B5) An allocation method as described in Appendix B1, including a presentation control process in which the at least one processor presents the evaluation results to a user, and a reception process in which the at least one processor receives a generative model specified by the user, wherein in the allocation process, the at least one processor allocates the generative model specified by the user to the target task.

[0143] (Appendix B6) An allocation method described in Appendix B5, wherein each of the multiple generative models has a person who is the basis for the generative model, and the at least one processor presents to the user relationship information indicating the relationship between each of the people who are the basis for the multiple generative models.

[0144] (Appendix B7) An allocation method described in any of Appendices B1 to B6, including an execution control process in which the at least one processor causes the generative model assigned to each step to generate an output in each of a plurality of steps included in the task, and an execution control process in which the second or subsequent steps cause the generative model to generate an output based on the output generated in the preceding step.

[0145] (Appendix B8) An allocation method as described in Appendix B7, including a presentation control process in which the at least one processor presents to a user a deliverable generated by the generative model in each of the plurality of processes, and a reception process in which the at least one processor receives an instruction to change the generative model to be allocated to each of the plurality of processes, wherein in the execution control process, the at least one processor generates a deliverable in at least one of the second or subsequent processes in the changed allocation based on a deliverable generated in a preceding process.

[0146] (Appendix C1) An allocation program that causes a computer to function as an evaluation result acquisition means that acquires evaluation results obtained by evaluating multiple generative models that have been machine-learned to execute a given task and generate a deliverable, and an allocation means that determines multiple generative models to allocate to a target task based on the evaluation results for the target task to be executed.

[0147] (Appendix C2) The allocation program described in Appendix C1, wherein the allocation means allocates to the target task a generative model, of the plurality of generative models for which the evaluation results have been obtained, whose evaluation results for the target task satisfy predetermined conditions.

[0148] (Appendix C3) The allocation program according to appendix C1 or C2, wherein the allocation means allocates generative models to the target task based on attribute information indicating attributes of each of the plurality of generative models.

[0149] (Appendix C4) The allocation program according to any one of appendices C1 to C3, wherein the allocation means allocates a generative model to each of a plurality of steps included in the target task based on an evaluation result for each of the plurality of steps.

[0150] (Appendix C5) An allocation program described in Appendix C1, which causes the computer to function as a presentation control means for presenting the evaluation results to a user and as a reception means for receiving a generative model specified by the user, and the allocation means allocates the generative model specified by the user to the target task.

[0151] (Appendix C6) An allocation program described in Appendix C5, wherein each of the multiple generative models has a person who is the basis for the generative model, and the presentation control means presents to the user relationship information indicating the relationship between each of the people who are the basis for the multiple generative models.

[0152] (Appendix C7) An allocation program according to any one of appendices C1 to C6, which causes the computer to function as an execution control means that causes the generative model allocated to each step in each of a plurality of steps included in the task to generate an output, and in at least one of the second and subsequent steps, generates an output based on the output generated in the preceding step.

[0153] (Appendix C8) An allocation program described in Appendix C7, which causes the computer to function as a presentation control means that presents to a user the deliverables generated by the generative model in each of the plurality of processes, and as a receiving means that receives an instruction to change the generative model to be allocated to each of the plurality of processes, and the execution control means generates a deliverable in at least one of the second or subsequent processes in the changed allocation based on the deliverable generated in the preceding process.

[0154] (Appendix D1) An information processing device comprising at least one processor, the at least one processor executing an evaluation result acquisition process that acquires evaluation results obtained by evaluating multiple generative models that have been machine-learned to execute a given task and generate a deliverable, and an allocation process that determines multiple generative models to be allocated to a target task to be executed based on the evaluation results for the target task.

[0155] The information processing device may further include a memory, and the memory may store a program for causing the at least one processor to execute each of the processes.

[0156] (Appendix D2) The information processing device described in Appendix D1, wherein in the allocation process, the at least one processor allocates to the target task a generative model, of the plurality of generative models for which the evaluation results have been obtained, whose evaluation results for the target task satisfy predetermined conditions.

[0157] (Appendix D3) The information processing device according to appendix D1 or D2, wherein in the allocation process, the at least one processor allocates generative models to the target task based on attribute information indicating attributes of each of the plurality of generative models.

[0158] (Appendix D4) The information processing device described in any of Appendices D1 to D3, wherein in the allocation process, the at least one processor allocates a generative model to each of the multiple steps included in the target task based on an evaluation result for each of the multiple steps.

[0159] (Appendix D5) The information processing device described in Appendix D1, wherein the at least one processor executes a presentation control process to present the evaluation results to a user and a reception process to receive a generative model specified by the user, and in the allocation process, allocates the generative model specified by the user to the target task.

[0160] (Appendix D6) An information processing device as described in Appendix D5, wherein each of the plurality of generative models has a person on which the generative model is based, and the at least one processor presents to the user relationship information indicating the relationships between the people on which the plurality of generative models are based.

[0161] (Appendix D7) An information processing device described in any of Appendices D1 to D6, wherein the at least one processor executes an execution control process for causing the generative model assigned to each step to generate an output in each of a plurality of steps included in the task, and for at least one of the second and subsequent steps, an output is generated based on the output generated in the preceding step.

[0162] (Appendix D8) The information processing device described in Appendix D7, wherein the at least one processor executes a presentation control process that presents to a user the deliverables generated by the generative model in each of the plurality of processes, and a reception process that receives an instruction to change the generative model to be allocated to each of the plurality of processes, and in the execution control process, in at least one of the second or subsequent processes in the changed allocation, a deliverable is generated based on the deliverable generated in the preceding process.

[0163] (Appendix E) A non-transient recording medium on which an allocation program is recorded that causes a computer to function as an evaluation result acquisition means that acquires evaluation results obtained by evaluating multiple generative models that have been machine-learned to execute a given task and generate a deliverable, and an allocation means that determines multiple generative models to allocate to a target task to be executed based on the evaluation results for the target task. [Explanation of symbols]

[0164] 1. Information processing equipment 101 Evaluation result acquisition unit (evaluation result acquisition means) 102 Allocation unit (allocation means) 1A Information processing equipment 101A evaluation result acquisition unit (evaluation result acquisition means) 102A Allocation unit (allocation means) 103A Reception section (reception means) 105A Execution control unit (execution control means) 106A Presentation control unit (presentation control means) 111A Generative Model 112A Evaluation Results

Claims

1. an evaluation result acquisition means for acquiring evaluation results obtained by evaluating a plurality of generative models that have been machine-learned to execute a given task and generate a deliverable; and an allocation means for determining a plurality of generative models to be allocated to a target task based on the evaluation result for the target task to be executed.

2. The information processing apparatus according to claim 1 , wherein the allocation means allocates to the target task a generative model whose evaluation result for the target task satisfies a predetermined condition, from among the plurality of generative models for which the evaluation result has been acquired.

3. The information processing apparatus according to claim 1 , wherein the allocation means allocates the generative models to the target task based on attribute information indicating attributes of each of the plurality of generative models.

4. 3. The information processing apparatus according to claim 1, wherein the allocation means allocates the target task to each of the plurality of steps based on an evaluation result for each of the plurality of steps included in the target task.

5. a presentation control means for presenting the evaluation results to a user; a receiving means for receiving a designation of a generation model by the user, The information processing apparatus according to claim 1 , wherein the allocation means allocates the generative model designated by the user to the target task.

6. Each of the plurality of generative models has a person on whose behalf the generative model is based; The information processing apparatus according to claim 5 , wherein the presentation control means presents to the user relationship information indicating relationships between the people on which the plurality of generative models are based.

7. 3. The information processing device according to claim 1, further comprising: an execution control means for causing the generative model assigned to each process to generate a deliverable in each of a plurality of processes included in the task, and for at least one of the second and subsequent processes, causing the deliverable to be generated based on a deliverable generated in a preceding process.

8. a presentation control means for presenting to a user the results generated by the generative model in each of the plurality of steps; a receiving means for receiving an instruction to change the generative model to be allocated to each of the plurality of processes, 8. The information processing apparatus according to claim 7, wherein said execution control means causes at least one of the second and subsequent steps in the changed allocation to generate a product based on a product generated in a preceding step.

9. At least one processor An evaluation result acquisition process for acquiring evaluation results of a plurality of generative models trained by machine learning to execute a given task and generate a deliverable; an allocation process for determining a plurality of generative models to be allocated to a target task based on the evaluation result for the target task to be executed.

10. Computer, An evaluation result acquisition means for acquiring an evaluation result obtained by evaluating a plurality of generative models that have been machine-learned to execute a given task and generate a deliverable; and An allocation program that functions as an allocation means that determines a plurality of generative models to be allocated to a target task based on the evaluation results for the target task to be executed.

Images