Prediction program, prediction method, and information processing device

Abstract

This information processing device inputs first data to a first machine learning model and calculates a first feature amount. The information processing device identifies information for updating a parameter of the first machine learning model on the basis of the first feature amount. The information processing device updates the parameter of the first machine learning model using the information to be updated. The information processing device outputs a prediction result obtained by inputting the first data to the first machine learning model in which the parameter has been updated.

Description

Prediction program, prediction method, and information processing device

[0001] The present invention relates to a prediction program, a prediction method, and an information processing device.

[0002] Specialist LLMs (Large Language Models) specialized in each field are generated by additionally training general-purpose LLMs (LLMs) that have been trained using general training data. For example, a general-purpose LLM is additionally trained using training data related to mathematics to generate a mathematics-specific LLM, and a general-purpose LLM is additionally trained using training data related to program coding to generate a coding-specific LLM.

[0003] An information processing device is known that uses the multiple expert LLMs generated in this way to make predictions for data to be predicted. For example, the information processing device inputs the data to be predicted into a classifier, assigns the data to an appropriate expert LLM based on the results of the classifier, and makes predictions using the expert LLMs.

[0004] JP 2023-73095 A

[0005] However, the above technique requires the use of a classifier, which increases the amount of space required for calculation, such as the need for training data to generate the classifier, memory capacity to store the classifier, and retraining the classifier every time the number of expert LLMs is increased.

[0006] In one aspect, an object of the present invention is to provide a prediction program, a prediction method, and an information processing device that can reduce the amount of spatial calculation.

[0007] In a first proposal, the prediction program is characterized in that it causes a computer to execute a process of inputting first data into a first machine learning model to calculate a first feature, identifying information for updating parameters of the first machine learning model based on the first feature, updating the parameters of the first machine learning model using the updating information, and inputting the first data into the first machine learning model with the updated parameters to output the prediction result obtained.

[0008] According to one embodiment, the space complexity can be reduced.

[0009] FIG. 1 is a diagram illustrating an information processing device according to a first embodiment. FIG. 2 is a diagram illustrating a reference technology. FIG. 3 is a functional block diagram illustrating a functional configuration of an information processing device according to a first embodiment. FIG. 4 is a diagram illustrating a difference information DB. FIG. 5 is a diagram illustrating generation of the difference information DB. FIG. 6 is a diagram illustrating a calculation process during prediction. FIG. 7 is a diagram illustrating an update process during prediction. FIG. 8 is a diagram illustrating a prediction process during prediction. FIG. 9 is a flowchart illustrating a flow of a pre-learning process. FIG. 10 is a flowchart illustrating a flow of a prediction process. FIG. 11 is a diagram illustrating an example of processing using multiple weight differences. FIG. 12 is a diagram illustrating another example of prediction processing using an updated general-purpose LLM. FIG. 13 is a diagram illustrating an example of updating a general-purpose LLM. FIG. 14 is a diagram illustrating an example of a hardware configuration.

[0010] The following describes in detail embodiments of the prediction program, prediction method, and information processing device disclosed herein with reference to the accompanying drawings. Note that the present invention is not limited to these embodiments. The embodiments can be combined as appropriate within a consistent range.

[0011] (Explanation of Information Processing Apparatus) Fig. 1 is a diagram illustrating an information processing apparatus 10 according to Example 1. The information processing apparatus 10 illustrated in Fig. 1 is an example of a computer that selects an appropriate large language model (LLM) for an input prompt, which is an example of data to be predicted, and predicts the next token using the selected large language model.

[0012] In general, when multiple trained expert LLMs (LLMs) exist that are specifically trained for a particular domain, overall accuracy can be improved by using the most appropriate expert LLM for the input prompt. For example, a math prompt can be routed to an LLM that has been trained specifically for math, thereby improving accuracy.

[0013] Here, a reference technique for routing will be described. FIG. 2 is a diagram illustrating the reference technique. As shown in FIG. 2, in the reference technique, a general-task LLM trained using general training data, a mathematics-specific LLM generated by performing additional training on the general-task LLM using mathematics-related training data, and a coding-specific LLM generated by performing additional training on the general-task LLM using coding-related training data are stored in the memory of a GPU (Graphics Processing Unit). The GPU's memory also stores a classifier that predicts the LLM to be used for an input prompt, which has been trained as a router to predict which model is likely to have a high accuracy rate for a prompt.

[0014] Consider the prediction process for the prompt "If x + y = 22 and x = 12, what is y?" in this situation. In this case, the prompt "If x + y = 22 and x = 12, what is y?" is input to the classifier, and after the classifier determines the destination to be the mathematics-specialized LLM, the classifier inputs the prompt to the mathematics-specialized LLM. The mathematics-specialized LLM then outputs a prediction result in response to the input prompt.

[0015] As described above, in the reference technology, a dedicated routing model (classifier) ​​is used for routing, but the cost of training the routing model and the amount of space required for deploying the routing destination model are high. For example, a huge amount of training data is required to generate the classifier, and the classifier needs to be retrained every time the number of routing destination expert LLMs increases, and GPU memory is required to store the classifier and multiple expert LLMs.

[0016] Therefore, the information processing device 10 according to the first embodiment uses the embedding feature of the general-purpose LLM itself to calculate what kind of data the input prompt is similar to in terms of the embedding feature, and predicts the next token by changing the weight of the general-purpose LLM so that it approaches the weight of the expert LLM (specialized LLM) associated with similar data.

[0017] Specifically, as shown in FIG. 1 , the information processing device 10 maintains a difference information DB that associates the feature of a prompt for each expert LLM with the difference in network weights (parameters) between each expert LLM and a general-purpose LLM. In this state, the information processing device 10 inputs a prompt, which is an example of first data to be predicted, into a general-purpose LLM, which is an example of a first machine learning model, to calculate embedded features. The information processing device 10 then searches the difference information DB for features similar to the calculated embedded features and identifies the difference information for the general-purpose LLM. The information processing device 10 then updates the parameters of the general-purpose LLM using the identified difference information. The information processing device 10 then inputs the prompt into the general-purpose LLM with the updated parameters, predicts, and outputs the next token.

[0018] In this way, the information processing apparatus 10 according to the first embodiment uses the general-purpose LLM itself for domain identification, and reduces the amount of spatial calculation by changing the weights (parameters) of the network of the general-purpose LLM itself.

[0019] 3 is a functional block diagram showing the functional configuration of the information processing device 10 according to Example 1. As shown in FIG. 3, the information processing device 10 includes a communication unit 11, a storage unit 12, and a control unit 20.

[0020] The communication unit 11 is a processing unit that controls communication with other devices, and is realized by, for example, a communication interface, etc. For example, the communication unit 11 receives a prompt, which is an example of data to be predicted, from a management terminal used by an administrator, and transmits the prediction result to the management terminal.

[0021] The storage unit 12 is a processing unit that stores various data and programs executed by the control unit 20, and is realized by, for example, a memory, a hard disk, etc. The storage unit 12 stores a general-purpose LLM 13 and a difference information DB 14.

[0022] The general-purpose LLM 13 is a large-scale language model generated using general training data. Specifically, the general-purpose LLM 13 is a general-purpose LLM that is not specialized for a specific domain and is suitable for various natural language processing tasks such as text classification, sentiment analysis, information extraction, text summarization, text generation, and question answering. Generally, an expert LLM specialized for a specific domain is generated by fine-tuning (additional learning) the general-purpose LLM.

[0023] The difference information DB 14 is a database that stores difference information relating to the difference between the weights of the networks of the multiple expert LLMs and the weights of the network of the general-purpose LLM, in association with each feature. That is, the difference information DB 14 stores the difference between the parameters of the trained expert LLMs and the parameters of the trained general-purpose LLMs.

[0024] 4 is a diagram illustrating the difference information DB 14. As shown in FIG. 4, the difference information DB 14 stores a "feature" indicating an embedded feature obtained from a prompt and a "difference" indicating difference information between the weight of the expert LLM and the weight of the general-purpose LLM, in association with each other.

[0025] 4 shows that the feature of "math prompt" is associated with "Difference A," which is the difference between a math-specific LLM and a general-purpose LLM that can appropriately predict a math prompt. Similarly, the feature of "chord prompt" is associated with "Difference B," which is the difference between a coding-specific LLM and a general-purpose LLM that can appropriately predict a coding prompt.

[0026] The "difference" may be the network weight (parameter) itself, or may be compressed information obtained by compressing the difference. The difference information DB 14 may be generated by the pre-processing unit 30 (described later), or may be generated and stored in another device.

[0027] The control unit 20 is a processing unit that controls the entire information processing device 10, and is realized by, for example, a processor. The control unit 20 has a pre-processing unit 30 and a prediction processing unit 40. Note that the pre-processing unit 30 and the prediction processing unit 40 are realized by, for example, electronic circuits included in the processor or processes executed by the processor.

[0028] The pre-processing unit 30 is a processing unit that performs preparations before making a prediction for a new prompt. Specifically, the pre-processing unit 30 generates the difference information DB 14, receives difference information from an external device, and stores the difference information in the difference information DB 14.

[0029] For example, an example of generating differential information will be described using a prompt related to mathematics. FIG. 5 is a diagram illustrating the generation of differential information DB 14. As shown in FIG. 5, the pre-processing unit 30 acquires the "mathematics prompt" used in training the mathematics-specialized LLM during the machine learning phase in which additional learning is performed using the general-purpose LLM 13. It should be noted that the prompts are not limited to those used in training the mathematics-specialized LLM; prompts that are highly accurate when predicted by the mathematics-specialized LLM or prompts input to the mathematics-specialized LLM during prediction can also be used. Furthermore, the additional learning is not limited to using the general-purpose LLM 13; learning data from regular training of the mathematics-specialized LLM can also be used.

[0030] Next, the pre-processing unit 30 inputs the acquired "mathematics prompt" into the general-purpose LLM 13 and calculates features. For example, the pre-processing unit 30 calculates features from the Transformer of the general-purpose LLM 13. Here, the pre-processing unit 30 calculates, as features, "token embedding," "average of outputs of all layers of the Transformer block for all tokens of the input prompt," "average of outputs of the final layer of the Transformer block for all tokens of the input prompt," "average of outputs of all layers of the Transformer block for the last token of the input prompt," "average of outputs of the final layer of the Transformer block for the last token of the input prompt," etc.

[0031] The pre-processing unit 30 then calculates the difference between the network weights of the trained mathematics-specialized LLM and the network weights of the general-purpose LLM 13. For example, the pre-processing unit 30 may simply calculate the difference in network weights, or may compress the calculated weights. Specifically, when generating a mathematics-specialized LLM through additional training of the general-purpose LLM 13, the pre-processing unit 30 calculates the weights of the general-purpose LLM 13 and the mathematics-specialized LLM (expert model) because the model structures of the LLMs are the same, and can compress the number of parameters using singular value decomposition (SVD) or the like.

[0032] The pre-processing unit 30 then inputs the "mathematics prompt" into the general-purpose LLM 13, calculates the feature values, and associates the weight differences (or compressed information obtained by compressing the differences) with each other, and stores them in the difference information DB 14. In this way, the pre-processing unit 30 calculates the feature values ​​and difference information of the prompts (learning data) used in training each expert LLM, and stores them in the difference information DB 14.

[0033] Returning to FIG. 3, the prediction processing unit 40 includes a calculation unit 41, an update unit 42, and a prediction unit 43, and is a processing unit that executes prediction for a new prompt that is newly input data to be predicted.

[0034] The calculation unit 41 is a processing unit that calculates a first feature amount (a feature amount of the new prompt) by inputting a new prompt into the general-purpose LLM 13. Specifically, the calculation unit 41 calculates the feature amount when a new prompt is input into the general-purpose LLM 13 that serves as the source of additional learning for each expert LLM.

[0035] 6 is a diagram illustrating the calculation process during prediction. As shown in FIG. 6 , when the calculation unit 41 receives a new prompt, "When x + y = 22 and x = 12, what is y?", the calculation unit 41 inputs the received new prompt into the general-purpose LLM 13. Then, the calculation unit 41 calculates features (embedded features) of the new prompt from the general-purpose LLM 13 using a method similar to the method described in the pre-processing unit 30. Thereafter, the calculation unit 41 outputs the calculated features and the new prompt to the update unit 42.

[0036] The update unit 42 is a processing unit that identifies information for updating the parameters of the general-purpose LLM 13 based on the feature quantities of the new prompt calculated by the calculation unit 41, and updates the parameters of the general-purpose LLM 13 using the identified information to be updated. Specifically, the update unit 42 identifies similar feature quantities similar to the feature quantities of the new prompt from among the feature quantities stored in the difference information DB 14, and identifies difference information associated with the similar feature quantities as information to be updated. Then, the update unit 42 adds the identified difference information to the parameters of the general-purpose LLM 13.

[0037] 7 is a diagram illustrating the update process during prediction. As shown in FIG. 7, the update unit 42 calculates the similarity between the feature of a new prompt and each feature stored in the difference information DB 14 using a common similarity calculation method such as cosine similarity or Euclidean distance. The update unit 42 then identifies the prompt with the highest similarity that is equal to or greater than a threshold, "What happens when x is multiplied 12 times when x = 12?", and obtains the "weight difference A" stored in association with this prompt.

[0038] The update unit 42 then adds the "weight difference A" to the network weight of the general-purpose LLM 13 to generate the updated general-purpose LLM 13. At this time, the update unit 42 does not perform the update if there are no prompts with a similarity greater than or equal to a threshold value or if there are only prompts whose Euclidean distance is greater than a certain value. The update unit 42 outputs the update result to the prediction unit 43.

[0039] The prediction unit 43 is a processing unit that outputs a prediction result obtained by inputting a new prompt into the updated general-purpose LLM 13 with updated weights. That is, the prediction unit 43 predicts the next token using the general-purpose LLM 13 to which the weight difference with the expert LLM trained with similar prompts (data) has been added.

[0040] 8 is a diagram illustrating the prediction process during prediction. As shown in FIG. 8, the prediction unit 43 inputs a new prompt, "When x + y = 22 and x = 12, what is y?" to the updated general-purpose LLM 13 in which the weight difference A has been added to the weight of the general-purpose LLM 13, and obtains the prompt, "y is 10," which is the result predicted by the updated general-purpose LLM 13. The prediction unit 43 then transmits the output result prompt, "y is 10," to a specified terminal or displays it on a display unit such as a monitor.

[0041] In this way, the prediction unit 43 does not separately invoke an expert LLM, but instead uses the general-purpose LLM 13 as a proxy for the expert LLM to perform predictions of new prompts.

[0042] (Flow of Pre-Learning Process) Next, the flow of the pre-learning process will be described. Here, as an example, an example will be described in which each expert LLM has already been trained by additional learning of the general-purpose LLM 13. However, the pre-learning process may also be performed by the pre-processing unit 30.

[0043] 9 is a flowchart showing the flow of the pre-learning process. As shown in FIG. 9, the pre-processing unit 30 acquires the learning data used in training the expert LLM (S101), inputs the learning data to the general-purpose LLM 13, and calculates the feature quantities (S102).

[0044] Next, the pre-processing unit 30 calculates the difference between the weight of the expert LLM and the weight of the general-purpose LLM 13 (S103), associates the feature amount with the difference information, and registers the result in the difference information DB 14 (S104). After that, if the pre-processing unit 30 continues the process (S105: No), it repeats S101 and subsequent steps, and when the generation of the corresponding difference information is completed (S105: Yes), it ends the process.

[0045] (Prediction Process Flow) Fig. 10 is a flowchart showing the flow of the prediction process. As shown in Fig. 10, when a new prompt is input (S201: Yes), the prediction processor 40 inputs the new prompt into the general-purpose LLM 13 and calculates a feature (S202). Next, the prediction processor 40 searches the difference information DB 14 for a similar feature that is similar to the calculated feature (S203).

[0046] If a similar feature equal to or greater than the threshold exists (S204: Yes), the prediction processor 40 acquires difference information corresponding to the retrieved similar feature (S205). The prediction processor 40 then adds the difference information to the general-purpose LLM 13 (S206), inputs a new prompt into the general-purpose LLM 13 after the addition, and predicts the next token (S207). The prediction processor 40 then outputs the prediction result (S208).

[0047] On the other hand, if there are no similar features equal to or greater than the threshold (S204: No), the prediction processing unit 40 inputs a new prompt to the general-purpose LLM 13 that has not yet performed addition, and predicts the next token (S209).Then, the prediction processing unit 40 outputs the prediction result (S210).

[0048] (Effect) As described above, the information processing device 10 uses the embedded features of the general-purpose LLM 13 itself to calculate what data the embedded input features are similar to, and predicts the next token by changing the weight of the general-purpose LLM 13 so that it approaches the weight of the specialized LLM associated with similar data.

[0049] Therefore, the information processing device 10 does not need to prepare an expert LLM or a classifier that performs routing. In other words, the information processing device 10 can substitute a classifier and multiple expert LLMs with a single general-purpose LLM 13, thereby reducing the amount of space calculation.

[0050] Furthermore, the information processing device 10 can reduce the memory capacity of the expert LLM and the classifier. Furthermore, since the information processing device 10 can use a large memory capacity for prediction processing, it is possible to suppress processing delays and reduce the processing load on the computer.

[0051] Furthermore, the information processing device 10 generates difference information including feature amounts and differences from the training data used to train the expert LLM. Therefore, the information processing device 10 can control the general-purpose LLM 13 to which the difference information has been added in the same way as an actual expert LLM, thereby suppressing a decrease in prediction accuracy.

[0052] Furthermore, the information processing device 10 stores the differential information in a compressed state in the DB and can add it to the weight while it is still compressed, thereby enabling a further reduction in memory capacity compared to when the differential information is stored as is.

[0053] Although the embodiments of the present invention have been described above, the present invention may be embodied in various different forms other than the above-described embodiments.

[0054] (Numeric Values, etc.) The machine learning model, the number of LLMs, the training data, the prompts, etc. used in the above examples are merely examples and can be changed as desired. The process flow described in each flowchart can also be changed as appropriate within a consistent range.

[0055] (Multiple Weighted Differences) For example, in the above example, the information processing device 10 acquires the weighted difference of the most similar feature from the difference information DB 14 and adds it to the general-purpose LLM 13. However, this is not limiting. For example, the information processing device 10 can update (expand or complement) the general-purpose LLM 13 by using multiple pieces of difference information.

[0056] FIG. 11 is a diagram illustrating an example of processing using multiple weighted differences. As shown in FIG. 11, the information processing device 10 references the difference information DB 14 to search for feature quantities similar to the input prompt. Here, the information processing device 10 searches for three pieces of difference information, "weighted difference 1," "weighted difference 2," and "weighted difference 3," whose similarity is equal to or greater than a threshold. In this case, the information processing device 10 can add the average and variance values ​​of "weighted difference 1," "weighted difference 2," and "weighted difference 3" to the general-purpose LLM 13. Note that the information processing device 10 may acquire any number of weighted differences in descending order of similarity.

[0057] As a result, the information processing device 10 can suppress a decrease in prediction accuracy as much as possible even when there is a large variation in weight information due to the influence of training data or the like.

[0058] (Another Example of Prediction Processing) For example, in the processing described in the first embodiment, the next prompt may be predicted instead of the final answer in one prediction. In such a case, the processing up to obtaining the final answer will be described.

[0059] 12 is a diagram illustrating another example of prediction processing using an updated general-purpose LLM 13. As shown in FIG. 12, the information processing device 10 inputs a new prompt into the "general-purpose LLM 13 to which the weighted difference A has been added" generated using the method of Example 1, and obtains "prompt X1" as a prediction result. Next, the information processing device 10 inputs "prompt X1" into the "general-purpose LLM 13 to which the weighted difference A has been added," and obtains "prompt X2" as a prediction result. Thereafter, the information processing device 10 repeats the prediction processing of inputting the prediction result into the "general-purpose LLM 13 to which the weighted difference A has been added" until an answer to the new prompt is obtained.

[0060] In this way, the information processing device 10 can obtain a final prediction result (token) by repeatedly using the general-purpose LLM 13 that has been updated (extended) once, thereby realizing a short prediction process time.

[0061] The information processing device 10 can also perform an update process each time a prompt is acquired as a prediction result. Fig. 13 is a diagram illustrating an example of updating the general-purpose LLM 13. As shown in Fig. 13, when a new prompt is input, the information processing device 10 identifies a difference A based on the similarity using the method of the first embodiment, inputs the new prompt into the general-purpose LLM 13 to which the difference A has been added, and acquires a prediction result "prompt X1."

[0062] Next, similar to the method of Example 1, the information processing device 10 inputs the prediction result "prompt X1" into the general-purpose LLM 13 to calculate a feature amount, and identifies the difference B between the feature amount and a feature amount similar to the calculated feature amount. After that, the information processing device 10 inputs "prompt X1" into the general-purpose LLM 13 to which the difference B has been added, and obtains the prediction result "prompt X2."

[0063] In this way, the information processing device 10 can repeat updating and prediction of the general-purpose LLM 13 in response to input prompts until a final prediction result is obtained, thereby improving prediction accuracy.

[0064] (System) The information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings may be changed arbitrarily unless otherwise specified.

[0065] Furthermore, the specific form of distribution or integration of the components of each device is not limited to that shown in the figure. For example, the pre-processing unit 30 and the prediction processing unit 40 may be integrated. That is, all or some of the components may be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions of each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware using wired logic.

[0066] Furthermore, all or any part of the processing functions performed by each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware using wired logic.

[0067] (Hardware) Fig. 14 is a diagram illustrating an example of a hardware configuration. As shown in Fig. 14, an information processing device 10 includes a communication device 10a, a hard disk drive (HDD) 10b, a memory 10c, and a processor 10d. The components shown in Fig. 14 are connected to each other via a bus or the like.

[0068] The communication device 10a is a network interface card or the like, and communicates with other devices. The HDD 10b stores programs and databases that operate the functions shown in FIG.

[0069] The processor 10d reads out a program that executes the same processes as the respective processing units shown in Fig. 3 from the HDD 10b or the like and loads it into the memory 10c, thereby operating a process that executes the respective functions described in Fig. 3 or the like. For example, this process executes the same functions as the respective processing units of the information processing device 10. Specifically, the processor 10d reads out a program that has the same functions as the pre-processing unit 30, the prediction processing unit 40, or the like from the HDD 10b or the like. Then, the processor 10d executes a process that executes the same processes as the pre-processing unit 30, the prediction processing unit 40, or the like.

[0070] In this way, the information processing device 10 operates as an information processing device that executes a prediction method by reading and executing a program. The information processing device 10 can also realize functions similar to those of the above-described embodiment by reading the program from a recording medium using a medium reading device and executing the read program. Note that the program in these other embodiments is not limited to being executed by the information processing device 10. For example, the above-described embodiment may also be applied in the same way when another computer or server executes the program, or when these computers cooperate to execute the program.

[0071] This program may be distributed via a network such as the Internet. Alternatively, this program may be recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, a magneto-optical disk (MO), or a digital versatile disk (DVD), and may be read out from the recording medium and executed by a computer.

[0072] REFERENCE SIGNS LIST 10 Information processing device 11 Communication unit 12 Storage unit 13 General-purpose LLM 14 Difference information DB 20 Control unit 30 Preliminary processing unit 40 Prediction processing unit 41 Calculation unit 42 Update unit 43 Prediction unit

Claims

1. A prediction program that causes a computer to execute the following processes: inputting first data into a first machine learning model to calculate first features; identifying information for updating parameters of the first machine learning model based on the first features; updating the parameters of the first machine learning model using the updating information; and inputting the first data into the first machine learning model with the updated parameters, and outputting the prediction results obtained.

2. The prediction program of claim 1, characterized in that it causes a computer to execute the following processes: inputting multiple pieces of second data suitable for each of multiple second machine learning models into the first machine learning model to calculate each feature; calculating the difference between the parameters of each of the multiple second machine learning models and the parameters of the first machine learning model; and generating a database that associates each feature with difference information regarding the difference between the parameters.

3. The prediction program described in claim 2, characterized in that the calculation process compresses the difference between the parameters of each of the multiple second machine learning models and the parameters of the first machine learning model, and associates compressed information obtained by compressing the difference as the difference information with each of the features.

4. The prediction program of claim 2 or 3, characterized in that the identifying process refers to the database and identifies, as the information to be updated, the difference information associated with a similar feature among the features that is similar to the first feature; the updating process adds the difference information to parameters of the first machine learning model; and the output process outputs the prediction result obtained by inputting the first data into the first machine learning model to which the difference information has been added.

5. The prediction program described in claim 4, characterized in that the identifying process identifies each piece of difference information associated with a plurality of similar features similar to the first feature as the information to be updated, and the updating process adds an average value of each piece of difference information to a parameter of the first machine learning model.

6. The prediction program described in claim 2, characterized in that each of the plurality of second machine learning models is a model generated by additional training of the first machine learning model that has already been trained, and the plurality of second data is data used for additional training of any of the plurality of second machine learning models.

7. The prediction program of claim 1, wherein the first machine learning model is a large-scale language model, the first data is a prompt, and the output process comprises repeatedly inputting the output result of the first machine learning model with updated parameters into the first machine learning model with updated parameters until an answer to the prompt is obtained.

8. The prediction program of claim 1, wherein the first machine learning model is a large-scale language model, the first data is a prompt, the calculating process inputs second data, which is a prediction result of the first machine learning model whose parameters have been updated, into the first machine learning model before the update to calculate a third feature, the identifying process identifies second information for updating the parameters of the first machine learning model based on the third feature, the updating process updates the parameters of the first machine learning model using the second information, and the outputting process outputs the prediction result obtained by inputting the second data into the first machine learning model updated by the second information.

9. A prediction method characterized by executing the following processes by a computer: inputting first data into a first machine learning model to calculate first features; identifying information for updating parameters of the first machine learning model based on the first features; updating the parameters of the first machine learning model using the updating information; and inputting the first data into the first machine learning model with the updated parameters, and outputting the prediction result obtained.

10. An information processing device comprising: a control unit that inputs first data into a first machine learning model to calculate a first feature; identifies information for updating parameters of the first machine learning model based on the first feature; updates the parameters of the first machine learning model using the updating information; and outputs a prediction result obtained by inputting the first data into the first machine learning model whose parameters have been updated.

Images