Information processing system and information processing method

Abstract

To enable efficient and accurate prompt tuning while maintaining the secrecy of data in each user of an LLM.SOLUTION: The information processing system 10 includes a storage device that holds a metric obtained by applying each prompt obtained by each client to a data set of the client, and an arithmetic device that calculates an integrated metric across the clients for each prompt based on the metric in each client held for each prompt, and generates an integrated prompt obtained by integrating each prompt based on each prompt and the integrated metric.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates generally to information processing, and more particularly to a technique that enables efficient and accurate prompt tuning while maintaining the confidentiality of data for each user of an LLM. [Background technology]

[0002] Recent advances in large-scale language models (LLMs) in the field of natural language processing are having a major impact on business. High-performance language models, such as GPT-4, have demonstrated outstanding capabilities in a wide range of tasks, including text generation, dialogue, summarization, and translation, and are expected to improve business efficiency and create new value.

[0003] While the above-mentioned LLM can be widely used, there are cases where the accuracy of the generated data will not be as desired by the user unless appropriate tuning (for example, re-learning or prompt tuning) is performed according to the application field, purpose, etc. Patent Document 1 proposes a conventional technique for tuning not only LLM but also machine learning models in general. Patent Document 1 discloses a technology related to a learning system that more accurately adjusts the weights of each site during federated learning. When federated learning is applied to LLM, each user who wants to solve the same task using LLM brings their own dataset and works together to tune the data.

[0004] The above technology relates to a medical data federated learning system including a server storing multiple global models and multiple clients, each storing one or more local models. The client has a model acquisition unit that acquires from the server a target global model, which is a global model to be used in federated learning, from the multiple global models; a global model evaluation unit that performs training using the target global model and local training data of the client, evaluates the trained target global model using local evaluation data to obtain a global model evaluation result, and transmits the global model evaluation result to the server; and a local model evaluation unit that uses the local evaluation data to evaluate a target local model, which is the same type as the target global model, from the one or more local models stored in the client, to obtain a local model evaluation result, and transmits the local model evaluation result to the server. The server relates to a medical data federated learning system including a weight adjustment unit that adjusts the weight of each client based on the global model evaluation result and the local model evaluation result received from each client. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2024-25744 Summary of the Invention [Problem to be solved by the invention]

[0006] As mentioned above, tuning is necessary to effectively utilize LLM in specialized fields. However, frequent retraining of the model (fine tuning) is difficult due to its high computational cost. Therefore, prompt tuning, which has low computational costs, is widely used. Regarding prompt tuning, a method has been proposed in which the prompts themselves are generated using LLM, and this tends to further improve computational cost and accuracy.

[0007] The above-mentioned prompt tuning automatically tunes prompts to improve metrics related to a task, but a dataset for measuring the metrics is required. However, such datasets are often managed by the businesses that perform the task and kept secret as know-how or sensitive business information. In such cases, even if an attempt is made to apply prompt tuning like the conventional technology, it is not possible to collect measurement data from each business, and effective tuning cannot be expected.

[0008] Therefore, the present invention has been made in consideration of the above problems, and aims to provide a technology that enables efficient and accurate prompt tuning while maintaining the confidentiality of data for each user of an LLM. [Means for solving the problem]

[0009] In order to solve the above problem, an information processing system according to one embodiment of the present invention is an information processing system that integrates prompts in large-scale language models of each client that constitutes a client group, and includes: a storage device that stores metrics obtained by applying the metrics to a data set of the client for each prompt obtained at each of the clients; a processing device that calculates an integrated metric for each prompt across the clients based on the metrics for each client that are stored for each prompt; and a processing device that executes a processing to generate an integrated prompt that integrates the prompts based on each prompt and the integrated metric. [Effects of the Invention]

[0010] According to the present invention, efficient and accurate prompt tuning is possible while maintaining the confidentiality of data for each user of the LLM. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a diagram illustrating a network configuration including an information processing system according to an embodiment. [Figure 2] FIG. 2 illustrates an example of a hardware configuration of an integrated server according to an embodiment. [Figure 3] FIG. 2 illustrates an example of a hardware configuration of a client according to an embodiment. [Figure 4] FIG. 1 illustrates an example of the concept of tuning a large-scale language model. [Figure 5] FIG. 1 is a diagram illustrating a conceptual example of conventional machine learning. [Figure 6] FIG. 1 is a diagram illustrating a conceptual example of federated learning. [Figure 7] 1 is a diagram illustrating a functional configuration and a processing flow of an information processing system according to an embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of a flow of an information processing method according to an embodiment. [Figure 9] FIG. 10 is a diagram illustrating an example of a flow of an information processing method according to an embodiment. [Figure 10] 10A and 10B are diagrams illustrating an example of generating an integrated prompt sentence in the embodiment. [Figure 11] FIG. 10 is a diagram showing an example of a screen in the embodiment. [Figure 12] 1 is a diagram illustrating a functional configuration and a processing flow of an information processing system according to an embodiment. [Figure 13] FIG. 10 is a diagram illustrating an example of a flow of an information processing method according to an embodiment. [Figure 14] FIG. 10 illustrates an example of an integrated prompt and example data according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] In the following description, a communication device may be one or more communication interface devices, which may be one or more homogeneous communication interface devices (e.g., one or more NICs (Network Interface Cards)) or two or more heterogeneous communication interface devices (e.g., a NIC and an HBA (Host Bus Adapter)).

[0013] In the following description, a "memory" refers to one or more memory devices, which are an example of one or more storage devices. At least one memory device in the memory may be a volatile memory device or a non-volatile memory device.

[0014] In the following description, a "storage device" may refer to one or more persistent storage devices, which are an example of one or more storage devices. A persistent storage device may typically be a non-volatile storage device, specifically, for example, a hard disk drive (HDD), a solid state drive (SSD), or a non-volatile memory express (NVMe) drive. A storage device may include a memory.

[0015] Furthermore, in the following description, a "computing device" may be a processor, specifically, one or more processor devices. The at least one processor device may typically be a microprocessor device such as a CPU (Central Processing Unit), but may also be another type of processor device such as a GPU (Graphics Processing Unit). The at least one processor device may be a single-core or multi-core. The at least one processor device may also be a processor core. The at least one processor device may also be a processor device in a broader sense, such as a hardware circuit that performs part or all of the processing (e.g., an FPGA (Field-Programmable Gate Array), a CPLD (Complex Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit)).

[0016] In the following description, processing may be described using a "program" as the subject. However, since a program is executed by a CPU to perform a predetermined process using a storage device and / or an interface device, etc., as appropriate, the subject of the process may also be the CPU (or a device such as a controller having a processor). A program may be installed in a device such as a computer from a program source. The program source may be, for example, a program distribution server or a computer-readable (e.g., non-transitory) recording medium. In the following description, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0017] In addition, in the following description, when describing elements of the same type without distinguishing between them, common parts of the reference symbols may be used, and when describing elements of the same type with distinction between them, reference symbols or element identifiers may be used. <Network configuration including information processing systems>

[0018] FIG. 1 is a diagram showing an example of a network configuration of an information processing system 10 according to an embodiment.

[0019] The information processing system 10 of this embodiment is a system that enables efficient and accurate prompt tuning while maintaining the confidentiality of data for each user of the LLM, and is an information processing system that integrates prompts in the large-scale language models, i.e., LLM2012, of each client 200 that make up a client group.

[0020] The integrated server 100, which is the main component of such an information processing system 10, holds metrics obtained by applying the metrics to the client 200's dataset (local dataset 2013) for each prompt obtained at each client 200, and performs the following processes: calculating integrated metrics across each client 200 for each prompt based on the metrics held for each client 200 for each prompt; and generating an integrated prompt that integrates each prompt based on each prompt and the integrated metrics.

[0021] Furthermore, the integrated server 100 constituting the information processing system 10 is communicatively connected to each of the clients 200 via an appropriate network N. Each of the clients 200 is a computer connected to the integrated server 100 and used by a person in charge of the LLM2012, which is a large-scale language model of the client, who wishes to appropriately perform prompt tuning in the LLM2012 and improve the accuracy of the LLM2012 in a specialized field or the like.

[0022] The integrated server 100 shown in Figures 1 and 2 includes a storage device 101 consisting of a non-volatile storage device such as a hard disk drive or an embedded multimedia card, an I / O 102 for connecting to an input device such as a keyboard and an output device such as a display, a memory 103 consisting of a volatile storage device such as a RAM (Random Access Memory), a communication device 104 connected to a network N and communicating with at least a client 200, and a CPU 105 that calls a program 1011 stored in the storage device 101 into the memory 103 to perform overall control of itself and perform various judgments, calculations, and control processing.

[0023] The functions implemented in the storage device 101 of the integrated server 100 include a local prompt collection unit 110, a local metric collection unit 111, a global metric calculation unit 112, an integrated prompt generation unit 113, an integrated prompt distribution unit 114, a local category name collection unit 115, a global category name calculation unit 116, and a composite example data generation unit 117 (see FIGS. 7 and 11). Details of each of these functions will be described later. The storage device 101 also holds at least an LLM 1012. This LLM 1012 is a model that performs the functions of calculating global metrics (integrated metrics), generating integrated prompts, calculating global categories (integrated categories), and generating example data. Therefore, it is assumed that machine learning corresponding to these functions has been performed in advance using predetermined learning data.

[0024] Note that part of the processing performed by CPU 105 when it executes program 1011 may be executed by another arithmetic device (for example, hardware such as ASIC or FPGA). Also, memory 103 may be a ROM (Read Only Memory), which is a non-volatile storage element. ROM stores unchanging programs (for example, BIOS). RAM is a high-speed, volatile storage element such as DRAM (Dynamic Random Access Memory), and temporarily stores program 1011 executed by CPU 105 and data used when program 1011 is executed.

[0025] Alternatively, the above input and output devices may be provided by a client 200 connected to the integrated server 100 via a network N. The program 1011 executed by the CPU 105 is provided to the integrated server 100 via removable media (CD-ROM, flash memory, etc.) or the network N, and is stored in a non-volatile storage device 101, which is a non-transitory storage medium.

[0026] Therefore, integrated server 100 reads data from removable media via I / O 102. Integrated server 100 is a computer system configured on one physical computer or on multiple logically or physically configured computers, and may operate on a virtual computer built on multiple physical computer resources. Integrated server 100 is preferably configured on a cloud, but may also be on-premise configured on a specific computer (hardware).

[0027] The network N connecting the integrated server 100 and the client 200 may be the Internet, a local area network (LAN), a wide area network (WAN), or a mobile phone network, but is not limited to these.

[0028] The client 200 connected to the network N has, for example, an LLM 2012 trained by a business operator or the like for its own task, and a local dataset 2013 used for prompt tuning, and responds to a request for the task by using the LLM 2012 to provide a predetermined answer. Specifically, such a client 200 is implemented in the form of a PC (Personal Computer), a tablet terminal, a smartphone, XR (Cross Reality) goggles, a voice dialogue system, or the like. The LLM 2012 held by each client 200 is the same for all clients, but the local dataset 2013 is individually acquired and managed by the business operator or the like and is kept confidential and not provided to the outside.

[0029] Note that the data transfer between the integrated server 100 and the client 200 may be performed according to, for example, the protocol of the API (Application Programming Interface). In that case, it is assumed that each device has already implemented the functions and configurations for executing each process of requests and responses by the API.

[0030] On the other hand, the client 200, similar to the integrated server 100, includes a storage device 201, I / O 202, a memory 203, a communication device 204, and a CPU 205. Such a client 200 can be, for example, a terminal used when a person in charge of an organization performing a predetermined task uses an LLM to improve the efficiency of the task. In this case, in addition to desk work in the office, a situation where the LLM 2012 of the client 200 answers useful advice and suggestions on necessary procedures, such as component assembly and inspection of work content, at a factory or a construction site can also be assumed. <Regarding Learning in LLM>

[0031] First, the learning method and stages of a typical LLM are explained with reference to Figure 4. Generally, when learning an LLM, the following methods can be appropriately adopted: pre-training (Step 1), fine-tuning (Step 2), RLHF (Reinforcement Learning from Human Feedback, Step 3), and prompting (Step 4). Among these, pre-training is self-supervised learning using large-scale data, allowing the LLM to acquire vocabulary, grammar, and world knowledge. Fine-tuning adapts the LLM to specific specialized fields by learning using labeled data, i.e., teacher data. RLHF aligns the LLM output to human values ​​through reinforcement learning using human feedback. Prompting inputs context sentences / examples into the LLM to prompt the emergence of specific functions, which can be considered pseudo-learning in context. This invention focuses on prompting, which has the lowest learning cost among the above learning methods, and employs technology that appropriately applies the concept of associative learning to it. <About traditional machine learning and federated learning>

[0032] Next, the concepts of conventional machine learning and federated learning will be explained based on Figures 5 and 6. In conventional machine learning, each client (clients 1 to n in Figure 5) provides their own data set to a central server, which then merges the data and generates and trains a model. On the other hand, in federated learning, each client (clients 1 to n in Figure 6) trains a local model using its own data set and provides the weight information of these local models to the central server. In this case, the central server merges the weight information and trains an integrated model. The central server then responds with this merged weight information to each client.

[0033] On the other hand, even if an attempt is made to combine the above-described conventional associative learning with prompt tuning, unlike the conventional associative learning that integrates model parameters (an example of importance), an integration process for the prompt, i.e., the text, is required, and associative learning cannot be applied as is. These problems can be solved by applying the information processing method of this embodiment, and by appropriately applying the concept of associative learning to prompt tuning, LLM tuning in a specific field can be made efficient and accurate. <Information processing method: Main flow>

[0034] Next, the processing flow of the information processing method of this embodiment will be described together with each functional unit in the integrated server 100 and the client 200. Fig. 7 is a diagram showing the functional configuration and processing flow of the information processing system of this embodiment. Fig. 8 is a diagram showing an example of the flow (client) of the information processing method of this embodiment, and Fig. 9 is a diagram showing an example of the flow (integrated server) of the information processing method of this embodiment.

[0035] Here, the concept is shown in which the integrated server 100 and the client 200 work together to perform Federated Prompt Learning, which combines federated learning and prompt auto-tuning, to generate prompts that improve the client 200's performance on a specific task in LLM2012. This allows multiple users (clients) to cooperate in generating integrated prompts, and in this case, it becomes possible to perform prompt tuning without sharing data sets that the clients manage individually, keeping them confidential. In other words, it is possible to improve the performance of LLM2012 in the relevant field while keeping highly confidential data confidential.

[0036] First, the local prompt collection unit 110 of the integrated server 100 initializes, for example, a template prompt or a predetermined prompt that it has stored in advance, and distributes the initial prompt to each client 200 (S20). Meanwhile, the client 200 receives the initial prompt distributed from the integrated server 100 (S10), and performs prompt tuning by the local prompt tuning unit 211 (S11). This prompt tuning by the local prompt tuning unit 211 is performed by appropriately adopting a known method that uses the prompt tuning function of the LLM 2012 of the client 200. The local prompt tuning unit 211, in S11, _k (k is an identification value that uniquely identifies the client, 1 to n) and sends this back to integrated server 100.

[0037] Meanwhile, the local prompt collection unit 110 of the integrated server 100 collects the local prompts t k and acquires n local prompts (S21). The local metric collector 111 of the integrated server 100 then distributes the n local prompts acquired in S21 to each client 200 as prompts to be evaluated (S22). In this case, the prompts to be evaluated that are distributed to each client are local prompts other than those generated by the client itself.

[0038] On the other hand, the local metric calculation unit 210 of the client 200 calculates the local prompt t k Regarding the local metric P, we used a local dataset 2013 that we manage ourselves. i (t k ) is calculated (S12). i (t k ) is calculated at the local prompt t kWe can appropriately adopt known methods, such as calculating the accuracy rate, which indicates how well the answers given to LLM2012 when a question with the following information is answered matches the correct answers in the local dataset 2013.

[0039] The local metric calculation unit 210 of the client 200 associates the local metric calculated in S12 with the local prompt obtained in S11 and transmits the result to the integrated server 100 (S13). As a result, the local metric collection unit 111 of the integrated server 100 obtains the local prompt t k For each, the local metric P obtained from each client 200 i (t k ) information will be obtained.

[0040] Therefore, the global metric calculation unit 112 of the integrated server 100 calculates the local metric collection unit 111 for each local prompt t k For each, the local metric P obtained from each client 200 i (t k ) and integrate the local prompt t k The integrated metric P(t k ) is calculated (S23). k ) is calculated, for example, at the local prompt t k Regarding the local metric P obtained from each of the N clients 200, i (t k ) or by calculating the average of N local metrics P i (t k ) can be appropriately adopted. k Regarding the integrated metric P(t k ) is calculated (S24).

[0041] Next, the integrated prompt generation unit 113 of the integrated server 100 generates an integrated prompt sentence to be input to the LLM 1012, that is, each local prompt t k The integrated metric P(t k ) based on the integrated prompt global In this case, the integrated prompt generating unit 113 generates an integrated prompt sentence (see integrated prompt sentence G1 in FIG. 10) in the following procedure. (Step 1) Local prompt k and the integrated metric P(t k ) set {t k , P(t k )}, t global List them in the integration prompt as potential candidates. (Step 2) Based on the information in (Criterion 1) above, integrate the prompts global Write an instruction to generate a new one in the integrated prompt statement.

[0042] Next, the integrated prompt generation unit 113 of the integrated server 100 inputs the integrated prompt sentence created in S25 to the LLM 1012, and generates the integrated prompt t global In this generation, for example, the integrated prompt is generated by applying the integrated metric for each prompt to the LLM 1012 as the weight of the prompt.

[0043] According to this, the integrated prompt global This allows for more efficient and accurate prompt tuning while maintaining the confidentiality of data for each user of the LLM. global is delivered from the integrated server 100 to each client 200. Each client 200 receives the integrated prompt t global This is given to the local prompt tuning unit 211 to perform prompt tuning (S14).

[0044] FIG. 11 shows an example of a screen G10 displayed on a display connected to the integrated server 100 via the I / O 102 or the communication device 104 during the above-described series of processes. The screen G10 is composed of selection objects G11-G15 for each client 200, an information display area G16 related to the client 200 selected by the selection objects G11-G15, an integration-related information display area G17, and a status display field G18. Among these, the information display area G16 displays values ​​such as the ID of the selected client 200, the history of local prompts, the local learning epoch (the number of local tunings), the local metric history, and field notification options. The integration-related information display area G17 displays information such as the number of communications between the client 200 and the integrated server 100 (i.e., the number of server communications), the integration prompt statement, and the integration prompt. The status display field G18 displays status information such as the addition or removal of a client 200. <Information processing method: Generation of example data>

[0045] Next, the integration prompt global This section will explain the generation of example data that enhances the effect of prompt tuning by providing the example data to the LLM 1012 in addition to the above. Fig. 12 is a diagram showing the functional configuration and processing flow of the information processing system in this embodiment regarding the generation of this example data, and Fig. 13 is a diagram showing an example flow of the information processing method in this embodiment.

[0046] Here, the local category estimation unit 212 of each client 200 estimates categories in the local dataset 2013 of that client 200, i.e., local categories (S30). This estimation is performed, for example, by extracting superordinate concepts (e.g., mathematics) of each piece of data (e.g., sets of questions and correct answers in the fields of quadratic functions, trigonometric functions, and linear algebra) included in the local dataset 2013 using the LLM 2012. Furthermore, the local category estimation unit 212 of each client 200 transmits information on the local categories estimated in S30 to the integrated server 100 (S31).

[0047] Meanwhile, the local category name collection unit 115 of the integrated server 100 acquires the local category information transmitted from each of the clients 200 and passes it to the global category name calculation unit 116. The global category name calculation unit 116 calculates an integrated category name based on the local category information, that is, a category name that spans the local data sets 2013 of each of the clients 200 (S32). This calculation corresponds to, for example, the process of providing the LLM 1012 of the integrated server 100 with information on each local category to extract a superordinate concept that spans those local categories.

[0048] Next, the composite example data generation unit 117 of the integrated server 100 generates example data (hereinafter, composite example data) to be attached to the integrated prompt by issuing an instruction to, for example, the LLM 1012 to generate a data set related to the integrated category (S33). In this case, the LLM 1012, for example, has been learning about generating composite example data for each integrated category, and can output composite example data by attaching information about the integrated category. The composite example data generation unit 117 displays the composite example data on a display or the like connected via the I / O 102 or the communication device 104, and presents it to the user. Screen G20 in FIG. 14 shows an example of the display of the composite example data G21 thus generated.

[0049] The integrated example data generation unit 117 of the integrated server 100 passes the integrated prompt generation statement and the composite example data to the integrated prompt generation unit 113. The integrated prompt generation unit 113 receives the integrated prompt generation statement and the composite example data from the integrated example data generation unit 117 and generates an integrated prompt by providing them to the LLM 1012 (S34). By adding the composite example data to the integrated prompt generation statement in this way and providing it to the LLM 1012, it is possible to generate an accurate integrated prompt.

[0050] The local categories mentioned above are generally extracted from the main features of local data and are not limited to category names in the narrow sense, but include various modifications and equivalent configurations within the spirit of the claims. Similarly, the integrated categories mentioned above are generally extracted from the main features of data and are not limited to category names in the narrow sense, but include various modifications and equivalent configurations within the spirit of the claims. The synthetic example data mentioned above may be synthesized using a generative AI model that generates images, videos, audio, etc. in addition to a large-scale language model, and may have a form different from the corresponding text.

[0051] As described above, the information processing system of this embodiment enables efficient and accurate prompt tuning while maintaining the confidentiality of data for each user of the LLM.

[0052] The present invention is not limited to the above-described embodiments and includes various modifications and equivalent configurations within the spirit of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to configurations including all of the described configurations. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, part of the configuration of each embodiment may be added, deleted, or replaced with other configurations. For example, the integrated server 100 may provide an administration user interface (UI) and input / output data related to maintenance or other purposes via the administration UI. Furthermore, the administration UI may be provided to an information processing terminal of an administrator (not shown).

[0053] Furthermore, the aforementioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole in hardware, for example by designing them as integrated circuits, or may be realized in software by a processor interpreting and executing a program that realizes each function.

[0054] Information such as programs, tables, and files that realize each function can be stored in a storage device such as a memory, a hard disk, or an SSD (Solid State Drive), or in a recording medium such as an IC card, an SD card, or a DVD.

[0055] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines that are necessary for implementation. In reality, it can be considered that almost all components are interconnected.

[0056] Furthermore, the various explanations above can be summarized as follows. The following summary may include supplementary explanations and explanations of variations of the above explanations. In the information processing system of this embodiment, the arithmetic device may further execute a process of distributing a prompt obtained from each of the clients to other clients in the client group other than the client in question, and a process of collecting metrics from each of the clients by applying each of the prompts originating from the client in question or the other clients to a data set of the client, and storing the collected metrics in the storage device.

[0057] This allows so-called local prompts generated by tuning on each client to be distributed to other clients, allowing each client to collect its metrics, i.e., local metrics. By integrating these local metrics, an integrated metric can be generated, and integrated prompts can be generated based on this integrated metric. This enables more efficient and accurate prompt tuning while maintaining the confidentiality of data for each user of the LLM.

[0058] Furthermore, in the information processing system of this embodiment, each of the clients may have a data set that is managed individually by the client, and may calculate a metric in the large-scale language model when a prompt obtained at the other client and distributed from the information processing system is applied to the data set, and may respond to the information processing system by associating the metric information with the target prompt.

[0059] This allows each client to calculate a local metric and respond with the local metric information resulting from the calculation. This allows for efficient generation of integrated metrics and integrated prompts. This ultimately enables more efficient and accurate prompt tuning while maintaining the confidentiality of data for each user of the LLM.

[0060] In addition, in the information processing system of this embodiment, when generating the integrated prompt, the calculation device may generate the integrated prompt by applying the integrated metric for each prompt to a large-scale language model for prompt integration as the importance of the prompt.

[0061] This allows for the generation of appropriate integrated prompts based on local prompts, thereby enabling more efficient and accurate prompt tuning while maintaining the confidentiality of data for each user of the LLM.

[0062] Furthermore, in the information processing system of this embodiment, the computing device may further perform a process of collecting estimation results of main features (e.g., categories) of the datasets of each of the clients, identifying integrated main features (e.g., integrated categories) that span the datasets of each of the clients based on the estimation results of each of the clients, and generating example data to be attached to the integrated prompt by giving an instruction to a predetermined generation AI model to generate a dataset related to the integrated main features.

[0063] This allows for example data to be added to the large-scale language model to further improve the accuracy of prompt tuning using integrated prompts, thereby enabling more efficient and accurate prompt tuning while maintaining the confidentiality of data for each user of the LLM. [Explanation of symbols]

[0064] 10: Information processing system, 100: Integrated server, 200: Client

Claims

1. 1. An information processing system for integrating prompts in a large-scale language model of each of a group of clients, the information processing system comprising: a storage device for storing metrics obtained by applying the prompts obtained for each of the clients to the client's data set; a computing device that executes a process of calculating an integrated metric across each of the clients for each of the prompts based on the metrics for each of the clients that are held for each of the prompts, and a process of generating an integrated prompt that integrates each of the prompts based on each of the prompts and the integrated metric; An information processing system comprising:

2. The computing device a process of distributing the prompt obtained by each of the clients to other clients in the client group; collecting metrics from each of the clients obtained by applying each of the prompts originating from that client or the other clients to that client's data set, and storing the collected metrics in the storage device; Further implementation of The information processing system according to claim 1 .

3. Each of the clients: The client has a data set that is managed individually by the client, calculates a metric in the large-scale language model when a prompt obtained at the other client and distributed from the information processing system is applied to the data set, associates the metric information with the target prompt, and responds to the information processing system. The information processing system according to claim 2 .

4. The computing device When generating the integrated prompt, the integrated metric for each prompt is applied to a large-scale language model for prompt integration as the importance of the prompt, thereby generating the integrated prompt. The information processing system according to claim 1 .

5. The computing device The method further executes a process of collecting estimation results of main features of the data sets of each of the clients, identifying integrated main features across the data sets of each of the clients based on the estimation results of each of the clients, and generating example data to be assigned to the integrated prompt by giving an instruction to a predetermined generation AI model to generate a data set related to the integrated main features. The information processing system according to claim 1 .

6. 1. An information processing method for integrating prompts in large-scale language models of each of a group of clients, the method comprising: maintaining in a storage device metrics obtained by applying the metrics to the client's data set for each prompt obtained for each of the clients; calculating an integrated metric across the clients for each of the prompts based on the metrics for each of the clients that are held for each of the prompts, and generating an integrated prompt that integrates the prompts based on the prompts and the integrated metric; An information processing method that uses a computer.

Images