Information processing system and information processing method

Abstract

An information processing system 10 comprises: a storage device that retains metrics obtained by applying each prompt obtained from each client to the client's data set; and an arithmetic device that calculates, for each prompt, an integrated metric across each client on the basis of metrics at each client that are retained for each prompt, and generates an integrated prompt by integrating each prompt on the basis of each prompt and the integrated metrics.

Description

Information processing system and information processing method

[0001] The present invention relates generally to information processing, and more particularly to a technique that enables efficient and accurate prompt tuning while maintaining the confidentiality of data for each user of an LLM.

[0002] Recent advances in large language models (LLMs) in the field of natural language processing are having a major impact on business. High-performance language models, such as GPT-4, have demonstrated outstanding capabilities in a wide range of tasks, including text generation, dialogue, summarization, and translation, and are expected to improve business efficiency and create new value.

[0003] While the LLM can be widely used, there are cases where the accuracy of the generated data will not be as desired by the user unless appropriate tuning (e.g., re-learning or prompt tuning) is performed according to the application field, purpose, etc. Patent Document 1 proposes a conventional technique for tuning not only LLMs but also machine learning models in general. Patent Document 1 discloses a technology related to a learning system that more accurately adjusts the weights of each site during federated learning. Note that when federated learning is applied to LLMs, each user who wants to solve the same task using LLMs brings their own dataset and cooperates to tune the system.

[0004] The above technology relates to a medical data federated learning system including a server storing multiple global models and multiple clients, each of which stores one or more local models. The client has a model acquisition unit that acquires from the server a target global model, which is a global model to be subjected to federated learning, from the multiple global models; a global model evaluation unit that performs training using the target global model and local training data of the client, evaluates the trained target global model using local evaluation data to obtain a global model evaluation result, and transmits the global model evaluation result to the server; and a local model evaluation unit that uses the local evaluation data to evaluate a target local model, which is a local model of the same type as the target global model, from the one or more local models stored in the client, to obtain a local model evaluation result, and transmits the local model evaluation result to the server. The server relates to a medical data federated learning system including a weight adjustment unit that adjusts the weight of each client based on the global model evaluation result and the local model evaluation result received from each client.

[0005] JP 2024-25744 A

[0006] As mentioned above, tuning is necessary to effectively utilize LLMs in specialized fields. However, frequent retraining of models (fine tuning) is difficult due to its high computational cost. Therefore, prompt tuning, which has low computational costs, is widely used. Regarding prompt tuning, a method has been proposed in which the prompts themselves are generated by the LLM, and there is a tendency for computational costs and accuracy to be further improved.

[0007] The above-mentioned prompt tuning automatically tunes prompts to improve metrics related to a task, but a dataset for measuring the metrics is required. However, such datasets are often managed by the businesses that perform the task and kept secret as know-how or sensitive business information. In such cases, even if an attempt is made to apply prompt tuning like the conventional technology, it is not possible to collect measurement data from each business, and effective tuning cannot be expected.

[0008] Therefore, the present invention has been made in consideration of the above problems, and aims to provide a technology that enables efficient and accurate prompt tuning while maintaining the confidentiality of data for each user of an LLM.

[0009] In order to solve the above problem, an information processing system according to one embodiment of the present invention is an information processing system that integrates prompts in large-scale language models of each client that constitutes a client group, and includes: a storage device that stores metrics obtained by applying the metrics to a data set of the client for each prompt obtained at each of the clients; a processing device that calculates an integrated metric for each prompt across the clients based on the metrics for each client that are stored for each prompt; and a processing device that executes a processing to generate an integrated prompt that integrates the prompts based on each prompt and the integrated metric.

[0010] According to the present invention, efficient and accurate prompt tuning is possible while maintaining the confidentiality of data for each user of the LLM.

[0011] 1 is a diagram illustrating a network configuration including an information processing system according to an embodiment. FIG. 2 is a diagram illustrating an example of the hardware configuration of an integrated server according to an embodiment. FIG. 3 is a diagram illustrating an example of the hardware configuration of a client according to an embodiment. FIG. 4 is a diagram illustrating an example of the concept of tuning a large-scale language model. FIG. 5 is a diagram illustrating an example of the concept of conventional machine learning. FIG. 6 is a diagram illustrating an example of the concept of federated learning. FIG. 7 is a diagram illustrating a functional configuration and a processing flow of an information processing system according to an embodiment. FIG. 8 is a diagram illustrating an example of the flow of an information processing method according to an embodiment. FIG. 9 is a diagram illustrating an example of the generation of an integrated prompt sentence according to an embodiment. FIG. 10 is a diagram illustrating an example of a screen according to an embodiment. FIG. 11 is a diagram illustrating a functional configuration and a processing flow of an information processing system according to an embodiment. FIG. 12 is a diagram illustrating an example of the flow of an information processing method according to an embodiment. FIG. 13 is a diagram illustrating an example of an integrated prompt and example data according to an embodiment.

[0012] In the following description, a communication device may be one or more communication interface devices, which may be one or more homogeneous communication interface devices (e.g., one or more NICs (Network Interface Cards)) or two or more heterogeneous communication interface devices (e.g., a NIC and an HBA (Host Bus Adapter)).

[0013] In the following description, a "memory" refers to one or more memory devices, which are an example of one or more storage devices. At least one of the memory devices may be a volatile memory device or a non-volatile memory device.

[0014] In the following description, a "storage device" may be one or more persistent storage devices, which are an example of one or more storage devices. A persistent storage device may typically be a non-volatile storage device, specifically, for example, a hard disk drive (HDD), a solid state drive (SSD), or a non-volatile memory express (NVMe) drive. The storage device may include a memory.

[0015] In the following description, a "computing device" may be a processor, specifically, one or more processor devices. The at least one processor device may typically be a microprocessor device such as a CPU (Central Processing Unit), but may also be another type of processor device such as a GPU (Graphics Processing Unit). The at least one processor device may be a single-core or multi-core. The at least one processor device may also be a processor core. At least one processor device may be a broad processor device such as a hardware circuit (e.g., FPGA (Field-Programmable Gate Array), CPLD (Complex Programmable Logic Device), or ASIC (Application Specific Integrated Circuit)) that performs part or all of the processing.

[0016] In the following description, processing may be described using a "program" as the subject. However, since a program is executed by a CPU to perform a predetermined process using a storage device and / or an interface device, etc., as appropriate, the subject of the process may also be the CPU (or a device such as a controller having a processor). A program may be installed in a device such as a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable (e.g., non-transitory) recording medium. In the following description, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0017] In the following description, when describing elements of the same type without distinguishing between them, common parts of the reference symbols will be used, and when describing elements of the same type with distinction between them, reference symbols or element identifiers will be used. <Regarding a network configuration including an information processing system>

[0018] FIG. 1 is a diagram illustrating an example of a network configuration of an information processing system 10 according to an embodiment.

[0019] The information processing system 10 of this embodiment is a system that enables efficient and accurate prompt tuning while maintaining the confidentiality of data for each user of the LLM, and is an information processing system that integrates prompts in the large-scale language models, i.e., LLM2012, of each client 200 that make up a client group.

[0020] The integrated server 100, which is the main component of such an information processing system 10, holds metrics obtained by applying the metrics to the client 200's dataset (local dataset 2013) for each prompt obtained at each client 200, and performs the following processes: calculating integrated metrics across each client 200 for each prompt based on the metrics at each client 200 held for each prompt; and generating an integrated prompt that integrates each prompt based on each prompt and the integrated metrics.

[0021] Furthermore, the integrated server 100 constituting the information processing system 10 is communicably connected to each of the clients 200 via an appropriate network N. Each of the clients 200 is a computer connected to the integrated server 100 and used by a person in charge of a specific field or the like who wishes to appropriately perform prompt tuning in the LLM 2012, which is the large-scale language model of the client 200, and thereby improve the accuracy of the LLM 2012.

[0022] The integrated server 100 shown in FIGS. 1 and 2 includes a storage device 101 configured as a nonvolatile storage device such as a hard disk drive or an embedded multimedia card, an I / O 102 for connecting to an input device such as a keyboard and an output device such as a display, a memory 103 configured as a volatile storage device such as RAM (Random Access Memory), a communication device 104 connected to a network N and communicating with at least a client 200, and a CPU 105 that loads a program 1011 stored in the storage device 101 into the memory 103 to perform overall control of the server itself and to perform various judgments, calculations, and control processes.

[0023] The functions implemented in the storage device 101 of the integrated server 100 include a local prompt collection unit 110, a local metric collection unit 111, a global metric calculation unit 112, an integrated prompt generation unit 113, an integrated prompt distribution unit 114, a local category name collection unit 115, a global category name calculation unit 116, and a composite example data generation unit 117 (see FIGS. 7 and 11 ). Details of each of these functions will be described later. The storage device 101 also holds at least an LLM 1012. This LLM 1012 is a model that performs the functions of calculating global metrics (integrated metrics), generating integrated prompts, calculating global categories (integrated categories), and generating example data. Therefore, it is assumed that machine learning corresponding to these functions has been performed in advance using predetermined learning data.

[0024] Note that part of the processing performed by the CPU 105 when it executes the program 1011 may be executed by another computing device (for example, hardware such as an ASIC or FPGA). The memory 103 may be a ROM (Read Only Memory), which is a non-volatile storage element. The ROM stores unchanging programs (for example, BIOS). The RAM is a high-speed, volatile storage element such as a DRAM (Dynamic Random Access Memory), and temporarily stores the program 1011 executed by the CPU 105 and data used when the program 1011 is executed.

[0025] Alternatively, the above input and output devices may be provided by a client 200 connected to the integrated server 100 via a network N. The program 1011 executed by the CPU 105 is provided to the integrated server 100 via removable media (CD-ROM, flash memory, etc.) or the network N, and is stored in a non-volatile storage device 101, which is a non-transitory storage medium.

[0026] Therefore, integrated server 100 reads data from removable media via I / O 102. Integrated server 100 is a computer system configured on one physical computer or on multiple logically or physically configured computers, and may operate on a virtual computer built on multiple physical computer resources. Integrated server 100 is preferably configured on the cloud, but may also be on-premise configured on a specific computer (hardware).

[0027] The network N connecting the integrated server 100 and the client 200 may be the Internet, a local area network (LAN), a wide area network (WAN), or a mobile phone network, but is not limited to these.

[0028] The clients 200 connected to the network N have, for example, an LLM 2012 that has been trained by an operator such as a business operator regarding its own tasks, and a local dataset 2013 used for prompt tuning, and respond to requests regarding the tasks with a predetermined response using the LLM 2012. Specifically, such clients 200 are implemented in the form of a PC (Personal Computer), a tablet terminal, a smartphone, XR (Cross Reality) goggles, a voice dialogue system, or the like. Furthermore, the LLM 2012 held by each client 200 is the same for all clients, but the local dataset 2013 is individually acquired and managed by the business operator or the like and is kept confidential and not provided to outside parties.

[0029] Note that data exchange between integrated server 100 and client 200 may be performed according to, for example, an API (Application Programming Interface) protocol. In this case, it is assumed that each device is pre-implemented with the functions and configurations for executing each process of request and response by the API.

[0030] On the other hand, the client 200, like the integrated server 100, includes a storage device 201, an I / O 202, a memory 203, a communication device 204, and a CPU 205. Such a client 200 can be, for example, a terminal used by an organization's personnel performing a specific task when using the LLM to improve the efficiency of that task. In this case, in addition to desk work in an office, it is also possible to imagine a situation in which the LLM 2012 of the client 200 provides useful advice and suggestions on necessary procedures for, for example, assembling parts or inspecting work content at a factory or construction site. <About learning in the LLM>

[0031] First, the learning method and stages of a typical LLM will be described with reference to FIG. 4 . Generally, when learning an LLM, the following methods can be appropriately adopted: pre-learning (Step 1), fine-tuning (Step 2), RLHF (Reinforcement Learning from Human Feedback, Step 3), and prompting (Step 4). Among these, pre-learning is self-supervised learning using large-scale data, allowing the LLM to acquire vocabulary, grammar, and world knowledge. Fine-tuning adapts the LLM to a specific specialized field by learning using labeled data, i.e., training data. RLHF aligns the LLM's output to human values ​​through reinforcement learning using human feedback. Furthermore, prompting involves inputting context sentences / examples into the LLM to prompt the emergence of specific functions, and can be considered pseudo-learning within the context. In this invention, we focus on prompting, which has the lowest learning cost among the above learning methods, and adopt a technology that appropriately applies the concept of associative learning to it. <About conventional machine learning and associative learning>

[0032] Next, the concepts of conventional machine learning and federated learning will be explained with reference to FIGS. 5 and 6. In conventional machine learning, each client (clients 1 to n in FIG. 5) provides their respective data sets to a central server, which then merges the data sets and generates and trains a model. In contrast, in federated learning, each client (clients 1 to n in FIG. 6) trains a local model using its own data set and provides the weight information of those local models to the central server. In this case, the central server merges the weight information and trains an integrated model. The central server then responds with this merged weight information to each client.

[0033] On the other hand, even if an attempt is made to combine the above-described conventional associative learning with prompt tuning, unlike the conventional associative learning that integrates model parameters (an example of importance), an integration process for the prompt, i.e., the text, is required, and associative learning cannot be applied as is. This problem is solved by applying the information processing method of this embodiment, and by appropriately applying the concept of associative learning to prompt tuning, LLM tuning in a specific field can be made efficient and accurate. <Information Processing Method: Main Flow>

[0034] Next, the processing flow of the information processing method of this embodiment will be described together with each functional unit of the integrated server 100 and the client 200. Fig. 7 is a diagram showing the functional configuration and processing flow of the information processing system of this embodiment. Fig. 8 is a diagram showing an example flow (client) of the information processing method of this embodiment, and Fig. 9 is a diagram showing an example flow (integrated server) of the information processing method of this embodiment.

[0035] Here, the concept is shown in which the integrated server 100 and the client 200 cooperate to perform federated prompt learning, which combines federated learning and automatic prompt tuning, to generate prompts that improve specific task performance in the LLM 2012 of the client 200. As a result, multiple users (clients) cooperate to generate integrated prompts, and in this case, it becomes possible to perform prompt tuning without sharing data sets that the clients individually manage, while keeping them confidential. In other words, it is possible to improve the performance of the LLM 2012 in the field in question while keeping highly confidential data confidential.

[0036] First, the local prompt collection unit 110 of the integrated server 100 initializes, for example, a template prompt or a predetermined prompt stored in advance, and distributes the initial prompt to each client 200 (S20). Meanwhile, the client 200 receives the initial prompt distributed from the integrated server 100 (S10), and performs prompt tuning by the local prompt tuning unit 211 (S11). This prompt tuning by the local prompt tuning unit 211 is performed by appropriately adopting a known method using the prompt tuning function of the LLM 2012 of the client 200. The local prompt tuning unit 211 obtains the local prompt t ＿ｋ (k is an identification value that uniquely identifies the client, ranging from 1 to n) and sends it back to the integrated server 100 .

[0037] Meanwhile, the local prompt collection unit 110 of the integrated server 100 collects the local prompts t ｋ and acquires n local prompts (S21). The local metric collector 111 of the integrated server 100 distributes the n local prompts acquired in S21 to each client 200 as prompts to be evaluated (S22). In this case, the prompts to be evaluated that are distributed to each client are local prompts other than those generated by the client itself.

[0038] On the other hand, the local metric calculation unit 210 of the client 200 calculates the local prompt t ｋ , using a local data set 2013 managed by the user, ｉ (t ｋ ) is calculated (S12). ｉ (t ｋ ) is calculated at the local prompt t ｋ A known method may be appropriately adopted, such as calculating the accuracy rate, which indicates the degree to which the answer given to the LLM 2012 when a question with the following attached parameter matches the correct answer in the local dataset 2013.

[0039] The local metric calculation unit 210 of the client 200 associates the local metric calculated in S12 with the local prompt obtained in S11 and transmits the result to the integrated server 100 (S13). As a result, the local metric collection unit 111 of the integrated server 100 obtains the local prompt t ｋ For each, the local metric P obtained from each client 200 ｉ (t ｋ ) information will be obtained.

[0040] Therefore, the global metric calculation unit 112 of the integrated server 100 calculates the local metric collection unit 111 for each local prompt t ｋ For each, the local metric P obtained from each client 200 ｉ (t ｋ ) and integrate the local prompt t ｋ The integrated metric P(t ｋ ) is calculated (S23). ｋ ) is calculated, for example, by ｋ , the local metric P obtained from each of the N clients 200 is ｉ (t ｋ ) or by calculating the average value of N local metrics P ｉ (tｋ ) can be appropriately adopted. ｋ Regarding the integrated metric P(t ｋ ) is calculated (S24).

[0041] Next, the integrated prompt generation unit 113 of the integrated server 100 generates an integrated prompt sentence to be input to the LLM 1012, i.e., each local prompt t ｋ The integrated metric P(t ｋ ) based on the integrated prompt t ｇｌｏｂａｌ In this case, the integrated prompt generating unit 113 generates an integrated prompt sentence (see integrated prompt sentence G1 in FIG. 10) in the following procedure: (Step 1) Generate a local prompt t ｋ and the integrated metric P(t ｋ ) set {t ｋ , P(t ｋ )}, t ｇｌｏｂａｌ (Step 2) Based on the information in (Criterion 1) above, list the integrated prompt t ｇｌｏｂａｌ Write an instruction to generate a new one in the integrated prompt statement.

[0042] Next, the integrated prompt generation unit 113 of the integrated server 100 inputs the integrated prompt sentence created in S25 to the LLM 1012, and generates the integrated prompt t ｇｌｏｂａｌ In this generation, for example, the integrated prompt is generated by applying the integrated metric for each prompt to the LLM 1012 as the weight of the prompt.

[0043] According to this, the integrated prompt ｇｌｏｂａｌ This allows for more efficient and accurate prompt tuning while maintaining the confidentiality of data for each user of the LLM. ｇｌｏｂａｌis delivered from the integrated server 100 to each client 200. Each client 200 receives the integrated prompt t ｇｌｏｂａｌ This is then given to the local prompt tuning unit 211 to perform prompt tuning (S14).

[0044] FIG. 11 shows an example of a screen G10 displayed on a display connected to the integrated server 100 via the I / O 102 or the communication device 104 during the above series of processes. The screen G10 is composed of selection objects G11 to G15 for each client 200, an information display area G16 related to the client 200 selected by the selection objects G11 to G15, an integration-related information display area G17, and a status display field G18. Of these, the information display area G16 displays values ​​such as the ID of the selected client 200, local prompt history, local learning epoch (number of local tunings), local metric history, and field notification options. The integration-related information display area G17 displays information such as the number of communications between the client 200 and the integrated server 100, i.e., the number of server communications, the integration prompt sentence, and the integration prompt. The status display field G18 displays status information, such as the addition or removal of a client 200. <Information Processing Method: Generation of Exemplary Data>

[0045] Next, the integration prompt ｇｌｏｂａｌ The following describes the generation of example data that enhances the effect of prompt tuning by providing the example data to the LLM 1012 in addition to the above. Fig. 12 is a diagram showing the functional configuration and processing flow of the information processing system in this embodiment regarding the generation of this example data, and Fig. 13 is a diagram showing an example flow of the information processing method in this embodiment.

[0046] Here, the local category estimation unit 212 of each client 200 estimates the categories in the local data set 2013 of that client 200, i.e., the local categories (S30). This estimation is performed, for example, by extracting superordinate concepts (e.g., mathematics) of each piece of data (e.g., sets of questions and correct answers in the fields of quadratic functions, trigonometric functions, and linear algebra) included in the local data set 2013 using the LLM 2012. Furthermore, the local category estimation unit 212 of each client 200 transmits information about the local categories estimated in S30 to the integrated server 100 (S31).

[0047] Meanwhile, the local category name collection unit 115 of the integrated server 100 acquires the local category information transmitted from each of the clients 200 and passes it to the global category name calculation unit 116. The global category name calculation unit 116 calculates an integrated category name based on the local category information, that is, a category name that spans the local data sets 2013 of each of the clients 200 (S32). This calculation corresponds to, for example, the process of providing the LLM 1012 of the integrated server 100 with information on each local category and extracting a superordinate concept that spans those local categories.

[0048] Next, the composite example data generation unit 117 of the integrated server 100 generates example data (hereinafter, "composite example data") to be assigned to the integrated prompt by issuing an instruction to, for example, the LLM 1012 to generate a data set related to the integrated category (S33). In this case, the LLM 1012, for example, has been learning about generating composite example data for each integrated category, and can output composite example data by assigning information about the integrated category. The composite example data generation unit 117 displays the composite example data on a display or the like connected via the I / O 102 or the communication device 104, and presents it to the user. Screen G20 in FIG. 14 shows an example of the composite example data G21 thus generated.

[0049] The integrated example data generation unit 117 of the integrated server 100 passes the integrated prompt generation sentence and the integrated example data to the integrated prompt generation unit 113. The integrated prompt generation unit 113 receives the integrated prompt generation sentence and the integrated example data from the integrated example data generation unit 117 and generates an integrated prompt by adding them to the LLM 1012 (S34). By adding the integrated example data to the integrated prompt generation sentence and adding them to the LLM 1012 in this way, it is possible to generate an accurate integrated prompt.

[0050] Note that the local categories described above generally extract the main features of local data and are not limited to category names in the narrow sense, but include various modifications and equivalent configurations within the spirit of the claims. Similarly, the integrated categories described above generally extract the main features of data and are not limited to category names in the narrow sense, but include various modifications and equivalent configurations within the spirit of the claims. Furthermore, the synthetic example data described above may be synthesized using a generative AI model that generates images, videos, audio, etc. in addition to a large-scale language model, and may have a form different from the corresponding text.

[0051] As described above, the information processing system of this embodiment enables efficient and accurate prompt tuning while maintaining the confidentiality of data for each user of the LLM.

[0052] The present invention is not limited to the above-described embodiments and includes various modifications and equivalent configurations within the spirit of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to configurations including all of the described configurations. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, part of the configuration of each embodiment may be added, deleted, or replaced with another configuration. For example, the integrated server 100 may provide an administration user interface (UI), and maintenance or other related data may be input and output via the administration UI. Furthermore, the administration UI may be provided to an administrator's information processing terminal (not shown).

[0053] Furthermore, the aforementioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole in hardware, for example by designing them as integrated circuits, or may be realized in software by having a processor interpret and execute a program that realizes each function.

[0054] Information such as programs, tables, and files that realize each function can be stored in a storage device such as a memory, hard disk, or SSD (Solid State Drive), or in a recording medium such as an IC card, SD card, or DVD.

[0055] In addition, the control lines and information lines shown are those that are considered necessary for explanation, and do not necessarily represent all the control lines and information lines that are necessary for implementation. In reality, it can be assumed that almost all components are interconnected.

[0056] Furthermore, the various explanations above can be summarized as follows. The following summary may include supplementary explanations and explanations of variations of the above explanations. In the information processing system of this embodiment, the arithmetic device may further execute a process of distributing a prompt obtained from each of the clients to other clients in the client group other than the client in question, and a process of collecting metrics from each of the clients by applying each of the prompts originating from the client in question or the other clients to a data set of the client, and storing the collected metrics in the storage device.

[0057] This allows so-called local prompts generated by tuning on each client to be distributed to other clients, and their metrics, i.e., local metrics, can be collected from each client. By integrating these local metrics, an integrated metric can be generated, and integrated prompts can be generated based on this integrated metric. This enables more efficient and accurate prompt tuning while maintaining the confidentiality of data for each user of the LLM.

[0058] Furthermore, in the information processing system of this embodiment, each of the clients may have a data set that is managed individually by the client, and may calculate a metric in the large-scale language model when a prompt obtained at the other client and distributed from the information processing system is applied to the data set, and may respond to the information processing system by associating the metric information with the target prompt.

[0059] This allows each client to calculate a local metric and respond with the local metric information resulting from the calculation. This allows for efficient generation of integrated metrics and integrated prompts. This ultimately enables more efficient and accurate prompt tuning while maintaining the confidentiality of data for each user of the LLM.

[0060] In addition, in the information processing system of this embodiment, when generating the integrated prompt, the calculation device may generate the integrated prompt by applying the integrated metric for each prompt to a large-scale language model for prompt integration as the importance of the prompt.

[0061] This allows for the generation of appropriate integrated prompts based on local prompts, thereby enabling more efficient and accurate prompt tuning while maintaining the confidentiality of data for each user of the LLM.

[0062] In addition, in the information processing system of this embodiment, the computing device may further perform a process of collecting estimation results of main features (e.g., categories) of the datasets of each of the clients, identifying integrated main features (e.g., integrated categories) that span the datasets of each of the clients based on the estimation results of each of the clients, and generating example data to be attached to the integrated prompt by giving an instruction to a predetermined generation AI model to generate a dataset related to the integrated main features.

[0063] This allows for example data to be added to the large-scale language model to further improve the accuracy of prompt tuning using integrated prompts, thereby enabling more efficient and accurate prompt tuning while maintaining the confidentiality of data for each user of the LLM.

[0064] 10: Information processing system, 100: Integrated server, 200: Client

Claims

1. An information processing system that integrates prompts in large-scale language models of each client that makes up a client group, comprising: a storage device that stores metrics obtained by applying each prompt obtained at each of the clients to a data set of the client; a computing device that executes a process of calculating an integrated metric for each prompt across the clients based on the metrics for each of the clients stored for each of the prompts; and a process of generating an integrated prompt that integrates the prompts based on each of the prompts and the integrated metric.

2. The information processing system of claim 1, wherein the computing device further executes the following processes: a process of distributing prompts obtained from each of the clients to other clients in the client group other than the client in question; and a process of collecting metrics from each of the clients obtained by applying each of the prompts originating from the client in question or the other clients to the client's data set, and storing the collected metrics in the storage device.

3. The information processing system of claim 2, wherein each of the clients has a data set that is managed individually by the client, calculates a metric in the large-scale language model when a prompt obtained at the other client and distributed from the information processing system is applied to the data set, associates the metric information with the target prompt, and responds to the information processing system.

4. The information processing system according to claim 1, wherein the calculation device generates the integrated prompt by applying the integration metric for each prompt to a large-scale language model for prompt integration as the importance of the prompt.

5. The information processing system according to claim 1, wherein the computing device further executes a process of collecting estimation results of main features of each of the clients' datasets, identifying integrated main features across the datasets of each of the clients based on the estimation results of each of the clients, and generating example data to be attached to the integrated prompt by giving an instruction to a predetermined generative AI model to generate a dataset related to the integrated main features.

6. An information processing method for integrating prompts in large-scale language models of each of a client group, the information processing method being carried out by a computer, comprising the steps of: storing in a storage device metrics obtained by applying each prompt obtained by each of the clients to a data set of the client; calculating an integrated metric for each of the prompts across the clients based on the metrics for each of the clients stored for each of the prompts; and generating an integrated prompt that integrates each of the prompts based on each of the prompts and the integrated metric.

Images