Machine learning program, machine learning method, and information processing device

Abstract

To generate a prediction result that satisfies the required non-functional performance in a short period of time.SOLUTION: An information processing apparatus measures, for each of a plurality of data, non-functional performance representing performance for a requirement excluding a function of each of the plurality of data. The information processing apparatus, by machine learning that uses divided data obtained by dividing each of the plurality of data into a first portion of the data and a second portion of the data that is correct answer data as training data, executes machine learning processing of training a prediction model for predicting the second portion of the data in response to an input of the first portion of the data. The information processing apparatus uses a loss function including a parameter, which is a parameter determined according to a measurement result of the non-functional performance and which indicates a ratio of reflecting the non-functional performance in the prediction model, as a loss function in the machine learning processing.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a machine learning program, a machine learning method, and an information processing device. [Background technology]

[0002] Language models are known as a technology that supports program generation, document generation, and the like. For example, language models that perform automatic document generation are trained using a corpus, which is a large amount of language resource, to input a sequence of sentences up to the middle of the sentence and correctly predict the document that follows the input. Language models that perform automatic program generation are trained using a corpus, which is a large amount of language resource, to input a program prompt and correctly predict the code that follows the prompt.

[0003] To train such a language model, each time a code or word is generated, a classification task is solved to predict one code or word from all codes or words, the difference between the correct answer and the prediction is calculated as cross-entropy, and a loss function that minimizes the cross-entropy is used. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Greg Brockman, Mira Murati, Peter Welinder & OpenAI, [online], retrieved on February 4, 2020, “OpenAI API”, “https: / / openai.com / blog / openai-api / ” Summary of the Invention [Problem to be solved by the invention]

[0005] Meanwhile, programs and documents that are the targets of language model prediction have functional and non-functional performance, which respectively represent performance relative to functional and non-functional requirements. For example, in the case of a program, functional requirements are requirements that define the program's operation and behavior, while non-functional requirements are requirements excluding the functional requirements required of the program, such as the program's execution speed and the accuracy of the machine learning model generated by the program.

[0006] In training the language model, in order to generate a prediction result that achieves the desired non-functional performance, the generation of the prediction result and the learning of the language model must be repeated, which takes a long time. For example, the language model is generated by training based on a statistical approach, which is based on the superficial occurrence probability in the corpus. Therefore, when a language model is generated that depends on the non-functional performance status of each training data in the corpus and a prediction is performed using the language model, a prediction result that satisfies the desired non-functional performance may be generated immediately or may not be generated at all, which takes a very long time overall.

[0007] In one aspect, an object of the present invention is to provide a machine learning program, a machine learning method, and an information processing device that can generate prediction results that satisfy required non-functional performance in a short period of time. [Means for solving the problem]

[0008] In a first proposal, the machine learning program measures non-functional performance for each of a plurality of data, which represents performance for requirements excluding the functions of each of the plurality of data, and performs machine learning processing to train a predictive model that predicts the second part of the data in response to input of the first part of the data, using machine learning to train the split data obtained by dividing each of the plurality of data into a first part of the data and a second part that is correct answer data, and causes a computer to execute the processing, and uses a loss function in the machine learning processing that includes a parameter determined in response to the measurement results of the non-functional performance, the parameter indicating the proportion to which the non-functional performance is reflected in the predictive model. [Effects of the Invention]

[0009] In one aspect, predicted results that satisfy required non-functional performance can be generated in a short time. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating a language model of the information processing apparatus according to the first embodiment. [Figure 2] FIG. 2 is a diagram illustrating the training of a language model. [Figure 3] FIG. 3 is a diagram illustrating the reference technology. [Figure 4] FIG. 4 is a diagram illustrating the loss function used for training in the reference technology. [Figure 5] FIG. 5 is a diagram for explaining code generation using a language model of the reference technology. [Figure 6] FIG. 6 is a diagram illustrating the problems with training a language model in the reference technology. [Figure 7] FIG. 7 is a diagram illustrating training of a language model according to the first embodiment. [Figure 8] FIG. 8 is a functional block diagram of the information processing apparatus according to the first embodiment. [Figure 9] FIG. 9 is a diagram illustrating the measurement of non-functional performance. [Figure 10] FIG. 10 is a diagram illustrating the generation of training data. [Figure 11] FIG. 11 is a diagram illustrating machine learning of a language model. [Figure 12] FIG. 12 is a flowchart illustrating a process flow according to the first embodiment. [Figure 13] FIG. 13 is a diagram illustrating an example of a hardware configuration. DETAILED DESCRIPTION OF THE INVENTION

[0011] The following describes in detail the embodiments of the machine learning program, the machine learning method, and the information processing device disclosed in the present application, with reference to the accompanying drawings. Note that the present invention is not limited to these embodiments. Furthermore, the embodiments can be combined as appropriate within a consistent range. [Example]

[0012] (Description of information processing device) 1 is a diagram illustrating a language model of an information processing device 10 according to Example 1. The information processing device 10 illustrated in Fig. 1 is an example of a computer that generates a language model, which is an example of a prediction model that supports program generation, document generation, and the like.

[0013] For example, taking program generation as an example, in the learning phase, the information processing device 10 generates a language model using a corpus containing a large amount of language resources. In the generation phase, the information processing device 10 inputs a prompt q, which is an example of a seed for random number generation and indicates a starting point for code generation, into the machine-learned language model, and generates code c that follows the prompt q. As a result, the information processing device 10 can generate program code (script) in which the prompt q and code c are linked.

[0014] A language model is, for example, a model that gives the probability P(x) for a discrete symbol x (sequence x = x1 x2 x3...) in corpus D. x can be a word, a sentence, or a phoneme. When x is a word, P(x) is the probability that a language model M trained by machine learning using corpus D will predict and generate a sentence (or document). In other words, predicting a language model means finding the probability that a language model M trained based on corpus D will generate a sequence x, and the inherent properties of language are acquired through the constraints and ingenuity imposed on language model M during the learning process.

[0015] Next, the training of the language model will be described. FIG. 2 is a diagram illustrating the training of the language model. As shown in FIG. 2, the information processing device 10 inputs up to a middle of a document in a corpus into the language model, and obtains the generation result (prediction result) of the succeeding sentence (or word) of the input document from the language model. Then, the information processing device 10 updates various parameters of the language model so as to reduce the difference between the correct data and the prediction result. Note that various algorithms such as a neural network can be adopted for the language model.

[0016] For example, in the example of FIG. 2, the information processing device 10 generates a sequence t c1,1 and t c1,2 and data c1 (c1=t c1,1 ,t c1,2 ) is input to the language model, and the last sequence t of the input data is used as the prediction result of the language model. c1,2 The data c1´ (c1´=t´) contains the subsequent series c1,1 ,t´ c1,2 ,···t´ c1,n ) is acquired. Then, the information processing device 10 acquires the correct answer data c1 (c1=t c1,1 ,t c1,2 ,···t c1,n ) and the predicted result c1´(c1´=t´ c1,1 ,t´ c1,2 ,···t´ c1,n ) and update the various parameters of the language model so that the difference between them is small.

[0017] Here, n-gram and GPT (Generative Pre-Training) are known as reference technologies for commonly used language models. Figure 3 is a diagram explaining the reference technologies. n-Gram shown in Figure 3(a) is a model that expresses the words that follow the immediately preceding n-1 words using conditional probability. Specifically, n-Gram is a model that expresses the word sequence w1, w2, ... w i Given a word, the probability that word x appears after it is P(x|w1, w2,... w i ) is calculated using the previous n-1 words as a condition. For example, in the case of 2-gram, P(“Taro likes Hanako”|M 2_gram ) = p(is|Taro) p(Hanako|is) p(ga|Hanako) p(like|ga). M n_gram Machine learning in simply calculates the conditional probability for each token (word) in the corpus. Usually, the number of words is very large, and when n exceeds 5, most of the combinations become unknown.

[0018] GPT, shown in Figure 3(b), is an architecture that stacks Transformer decoders in multiple stages and is a model suitable for generative models with autoregressive properties, modeled using the probability of word occurrence. Transformer is a network architecture that combines an encoder and a decoder with an attention model. GPT performs machine learning using autoregressive properties, repeatedly inputting the output data output by the encoder in response to input data into the first decoder, and then inputting the output data of the first decoder into the second decoder.

[0019] Cross-entropy is used as the loss function for machine learning in this reference technology. Figure 4 is a diagram explaining the loss function used for training in the reference technology. As shown in Figure 4, in the reference technology, for each generated (predicted) word sequence, the difference between the generated (predicted) word sequence and the correct word sequence is calculated using the loss function shown in Equation (1), and machine learning is performed to minimize each difference. In the example of Figure 4, the correct data c gold Series 1 "t' gold,1 " and generated data c predicted Series 1 "t predicted,1 " and the difference between the correct data cgold Series 2 "t' gold,2 " and generated data c predicted Series 2 "t predicted,2 " is calculated using equation (1). A language model is then generated by machine learning that minimizes the sum of the differences between sequence 1 and sequence 2.

[0020]

number

[0021] Then, in the reference technology, a starting point such as a prompt is given to the language model calculated by the above process, and automatic generation such as program generation and document generation is performed. Figure 5 is a diagram explaining code generation using the language model of the reference technology. As shown in Figure 5, in the reference technology, the document data c to be predicted is new (t c,1 ,t c,2 ,t c,3 ,t c,4 ) is input to the language model, and the sequence (t c,4 ) is generated by the subsequent sequence of generated data c´ new (t´ c,1 ,t´ c,2 ,t´ c,3 ,t´ c,4 ,t´ c,5 ,···).

[0022] However, the language model of the reference technology is automatically generated based on a statistical approach based on the appearance probability (frequency, co-occurrence, etc.) of superficial characters in the corpus, and therefore cannot be automatically generated taking into account the non-functional performance of the code. Figure 6 is a diagram explaining the problems with training the language model of the reference technology.

[0023] As shown in Figure 6, the reference technology generates prompts "t1, t2, t3" from original code "t1, t2, t3, t4, t5," which is an example of program code, and generates training data with the prompts as explanatory variables and the original code as the objective variable. Then, the reference technology inputs the prompts into a language model to obtain the generated code (program code), and calculates the difference between the generated code and the original code as the loss function loss in Equation (1). deiff and train the language model so that the difference becomes small.

[0024] As such, in the reference technology, even if the original code, which is the input data, generates a program with a slow execution speed or a program that generates a machine learning model with low prediction accuracy, such information is not taken into account in training the language model. This is because the language model of the reference technology is primarily designed for general sentences, which do not have non-functional performance requirements like programs. In other words, in the reference technology, training is performed by uniformly imposing penalties without taking into account non-functional aspects within the corpus. Therefore, whether or not non-functional performance, such as generating a program with fast execution speed or high prediction accuracy, is achieved is not taken into account in program generation. If a program that achieves the required non-functional performance is not generated, it may be necessary to repeatedly generate prediction results and train the language model. This repeated generation of prediction results and training of the language model increases the overall time required to generate a program that achieves the required non-functional performance.

[0025] Therefore, the information processing device 10 according to the first embodiment adds a term equivalent to accuracy evaluation to a loss function during machine learning of a language model so as to generate an executable program with high non-functionality.

[0026] 7 is a diagram illustrating training of a language model according to the first embodiment. The information processing device 10 generates prompts "t1, t2, t3" from original code "t1, t2, t3, t4, t5" and generates training data with the prompts as explanatory variables and the original code as objective variables. Here, the information processing device 10 executes the original code using an execution environment, measures non-functional performance, and uses the measurement results to determine the proportion "α" at which the non-performance functions are reflected in the language model.

[0027] Then, the information processing device 10 inputs the prompt into the language model to obtain the generated code, calculates the difference between the generated code and the original code using a loss function loss that includes a parameter indicating the above ratio, and trains the language model so that the difference becomes smaller.

[0028] In this way, the information processing device 10 performs machine learning that takes into account non-functional performance, which is a characteristic required of a program, and can generate prediction results that satisfy the required non-functional performance in a short period of time without repeatedly generating prediction results and learning a language model.

[0029] (Functional configuration) 8 is a functional block diagram illustrating a functional configuration of the information processing device 10 according to Example 1. As shown in FIG. 8, the information processing device 10 includes a communication unit 11, a storage unit 12, and a control unit 20.

[0030] The communication unit 11 is a processing unit that controls communication with other devices, and is realized by, for example, a communication interface, etc. For example, the communication unit 11 receives various instructions from an administrator terminal, etc., and transmits the learning results to the administrator terminal.

[0031] The storage unit 12 is a processing unit that stores various data and programs executed by the control unit 20, and is realized by, for example, a memory, a hard disk, etc. The storage unit 12 stores a corpus 13, a training data DB 14, and a language model 15.

[0032] Corpus 13 is a database that stores a large amount of various data used to train a language model. For example, corpus 13 stores a plurality of programs (program codes), each of which includes a prompt and code following the prompt. In the above example, corpus 13 stores a large amount of original code.

[0033] The training data DB 14 is a database that stores training data for a language model. Specifically, the training data DB 14 stores a plurality of training data, which are divided data obtained by dividing each of a plurality of data into a first portion of the data and a second portion that is correct answer data. For example, each training data is supervised data in which a prompt and correct answer information (correct answer code) are associated with each other. The training data stored here may be generated using data stored in the corpus 13, or may be generated using other data.

[0034] The language model 15 is an example of a prediction model that predicts and outputs the subsequent part of input data. For example, in response to the input of a program prompt, the language model 15 generates code that follows the prompt and outputs the program code in which the prompt and code are concatenated. In another example, in response to the input of a document up to that point, the language model 15 generates the document from the middle of the document and outputs sentence data.

[0035] The control unit 20 is a processing unit that controls the entire information processing device 10, and is realized by, for example, a processor. The control unit 20 has a measurement unit 21, a training data generation unit 22, a machine learning unit 23, and a prediction unit 24. The measurement unit 21, the training data generation unit 22, the machine learning unit 23, and the prediction unit 24 are realized by electronic circuits included in the processor or processes executed by the processor.

[0036] The measurement unit 21 is a processing unit that measures non-functional performance, which indicates performance relative to requirements excluding the functions of each of the multiple data, for each of the multiple data stored in the corpus 13. The measurement unit 21 stores the measurement results in the memory unit 12 and outputs them to the training data generation unit 22. When each data is a program, the functional performance is information that defines the operation and behavior of the program, and the requirements excluding the functional requirements required for the program are non-functional requirements.

[0037] For example, an example will be described in which each of the multiple data stored in the corpus 13 is a script for generating a predictive model in machine learning. FIG. 9 is a diagram illustrating measurement of non-functional performance. As shown in FIG. 9, the measurement unit 21 executes code 1 to execute a prediction using a generated predictive model, and calculates the prediction accuracy at that time to be "0.83." The measurement unit 21 executes code 2 to execute a prediction using a generated predictive model, and calculates "NG" because the prediction accuracy at that time is below the threshold. The measurement unit 21 executes code 3 to execute a prediction using a generated predictive model, and calculates the prediction accuracy at that time to be "0.77." Note that the prediction accuracy can be determined by using, for example, the average value of each prediction using each data.

[0038] In the case of a program, not only the prediction accuracy but also the memory usage when the program is executed and the execution speed of the program can be used. In the case of the execution speed of the program, a function that converts a value from 0 to infinity into a value from 1 to 0 is used. For example, the measurement unit 21 may calculate the execution speed x as "x / (x+1)" or "x 2 / (x 2 +1) or arctan(x)×(2 / π).

[0039] However, the language model targeted in Example 1 is not limited to a model that generates a program. For example, a model that generates essays or Japanese language answers can also be targeted. In this case, when creating a model that generates sentences using sample answers in a situation where a large number of student answers are collected, such as in an XX preparatory school-sponsored or university entrance exam mock test, the model is generated so that answer samples with higher scores are more strongly reflected in the model. In this case, functional performance is a function that defines the direct use when using each of multiple answer data, such as the answer itself. Non-functional performance is a function that indirectly evaluates the direct function of each of multiple answer data, such as a score. Alternatively, non-functional performance in this case can also be considered an evaluation of the direct function of each of multiple answer data.

[0040] As another example, it can be used in a model that generates posts. In this case, functional performance is the content and number of characters of the post, and non-functional performance is the number of likes that indicate sympathy for the post.

[0041] 8, the training data generation unit 22 is a processing unit that generates training data using each of the multiple pieces of data stored in the corpus 13. Specifically, the training data generation unit 22 divides the data into a first part and a second part, generates training data in which the first part is an explanatory variable and the second part is a target variable (correct answer data), and stores the generated training data in the training data DB 14.

[0042] 10 is a diagram illustrating the generation of training data. As shown in FIG. 10, the training data generation unit 22 generates a training data set from a program code 1 “t 1,1 ,t 1,2 ,t 1,3 ,t 1,4 ,t 1,5 ,···t 1,n ", prompt 1_1 "t 1,1 ,t 1,2 ,t 1,3 " and the correct answer data "t 1,4 ,t 1,5 ,···t 1,n" Generate training data and prompt 1_2 "t 1,1 ,t 1,2 ,t 1,3 ,t 1,4 " and the correct answer data "t 1,5 ,···t 1,n In this way, the training data generating unit 22 generates training data such that the prompt 1_n "t 1,1 ,t 1,2 ,t 1,3 ,···,t 1,n-1 ", correct data "t 1,n Generate training data including "

[0043] Similarly, the training data generator 22 generates a program code 2 “t 2,1 ,t 2,2 ,t 2,3 ,t 2,4 ,t 2,5 ,···t 2,n ", prompt 2_1 "t 2,1 ,t 2,2 ,t 2,3 " and the correct answer data "t 2,4 ,t 2,5 ,···t 2,n " Generate training data and prompt 2_2 "t 2,1 ,t 2,2 ,t 2,3 ,t 2,4 " and the correct answer data "t 2,5 ,···t 2,n In this way, the training data generating unit 22 generates training data such that the prompt 2_n "t 2,1 ,t 2,2 ,t 2,3 ,···,t 2,n-1 ", correct data "t 2,n Generate training data including "

[0044] As described above, the training data generation unit 22 can generate training data using data stored in the corpus 13, thereby realizing efficient generation of training data and enabling accurate training data to be generated quickly.

[0045] The machine learning unit 23 is a processing unit that trains the language model 15 that predicts the second part of the data in response to input of the first part of the data, by machine learning using split data obtained by splitting the data into a first part of the data and a second part of the data, which is the correct answer data, as training data. At this time, the machine learning unit 23 uses, as a loss function, a loss function that includes a parameter indicating the proportion to which non-performance functions, determined in accordance with the measurement results of the non-performance functions, are reflected in the language model.

[0046] FIG. 11 is a diagram illustrating machine learning of the language model 15. As shown in FIG. 11, the machine learning unit 23 1,1 ,t 1,2 ,t 1,3 " is input to the language model 15, and the predicted result is "t' 1,4 ,t´ 1,5 ,···t´ 1,n Similarly, the machine learning unit 23 obtains prompt 1_2 “t 1,1 ,t 1,2 ,t 1,3 ,t 1,4 ,” is input to the language model 15, and the predicted result is “t´ 1,5 ,···t´ 1,n " to get

[0047] In this way, the machine learning unit 23 m,1 ,t m,2 ,t m,3 " is input to the language model 15 to obtain a prediction result, and the difference between the correct answer data and the prediction result is used to train the language model 15. For example, the machine learning unit 23 inputs the training data "prompt1_1(t 1,1 ,t 1,2 ,t 1,3 )+Correct code(t 1,4 ,t 1,5 ,···t 1,n )" and the predicted result "Prompt1_1(t 1,1 ,t 1,2 ,t 1,3 )+Correct code(t´ 1,4 ,t´ 1,5 ,···t´ 1,n)" to train the language model 15.

[0048] At this time, the machine learning unit 23 uses the non-functional performance-considered loss function shown in equation (2) to train the language model 15 using the difference between the correct answer data and the prediction result.

[0049]

number

[0050] In the loss function of formula (2), "λ × α" is a weight term that corresponds to a parameter indicating the proportion of non-performance functions reflected in the language model 15. "1-λ" is a loss term that corresponds to the difference between the correct data and the predicted result, and is a loss term that is based on the probability of occurrence of characters in each of multiple data. diff " is the cross-entropy shown in formula (1). Also, "α" is the measurement result of non-functional performance, and is a value measured by the measurement unit 21. "λ" is an adjustment parameter that indicates the degree to which non-functional performance is taken into consideration, and can be set arbitrarily. For example, "λ" is a coefficient that reflects superficial (literal) differences taking into consideration that not all code is necessarily executable, and when λ is 1, for example, functional performance is not reflected in the language model 15. When formula (2) is adopted as the loss function, a value that decreases as the non-functional performance increases is used for "α".

[0051] The prediction unit 24 is a processing unit that executes prediction processing using the language model 15 generated by the machine learning unit 23. For example, the prediction unit 24 inputs a prompt of a program into the language model 15, obtains a prediction result in which code following the prompt is generated, and can obtain the code of the program that includes the prompt and the code.

[0052] (Processing flow) Fig. 12 is a flowchart showing the flow of processing according to Example 1. As shown in Fig. 12, when the measurement unit 21 receives an instruction to start processing (S101: Yes), the measurement unit 21 acquires multiple programs from the corpus 13 (S102) and measures the non-functional performance of each of the multiple programs (S103).

[0053] Next, the training data generation unit 22 generates training data from multiple programs (S104). Then, the machine learning unit 23 predicts a chord from the prompt using each training data (S105), and executes machine learning of the language model 15 using the prediction result and a non-functional performance-aware loss function (S106).

[0054] (effect) As described above, the information processing device 10 collects a large number of scripts for creating machine learning models, executes them, and calculates prediction accuracy. The information processing device 10 generates pairs of prompts and generated programs from each program. For example, the information processing device 10 determines a minimum prompt length and generates pairs of prompts and target data that are longer than the minimum prompt length.

[0055] The information processing device 10 generates a program from each prompt using the language model 15, calculates a non-functional performance cross-entropy loss using the prediction result and correct answer data, and reflects it in the language model 15. In this way, the information processing device 10 adds a term equivalent to accuracy evaluation to the loss function during machine learning of the language model 15 so as to generate an executable program with high non-functionality, and therefore can generate prediction results that satisfy required non-functional performance in a short time.

[0056] In addition, by performing machine learning that takes into account the characteristics required of a program, the information processing device 10 can generate executable programs with high non-functional performance, such as execution speed and prediction accuracy, making it possible to develop software without repeated generation and trial and error.

[0057] Furthermore, the information processing device 10 can also perform machine learning using a loss function that does not use the term "1-λ" but uses only the weight term "λ×α", which corresponds to a parameter indicating the proportion of non-performance functions that are reflected in the language model 15. As a result, the information processing device 10 can easily generate a language model 15 that is specialized for non-performance functions.

[0058] Furthermore, the information processing device 10 can arbitrarily set the value of "λ" in equation (2), so that it can dynamically change whether to emphasize functional performance or non-performance functionality depending on the application of the model, and can provide a learning method that suits the use of the model.

[0059] Furthermore, the information processing device 10 can perform machine learning on not only programs but also document data, etc., and therefore can realize a highly versatile machine learning method. [Example]

[0060] Although the embodiments of the present invention have been described above, the present invention may be embodied in various different forms other than the above-described embodiments.

[0061] (Numbers, etc.) The program examples and training data examples used in the above embodiments are merely examples and can be changed as desired. Furthermore, the process flow described in each flowchart can also be changed as appropriate within a consistent range.

[0062] (system) The information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified.

[0063] Furthermore, the components of each device shown in the figure are functional concepts and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown. In other words, all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. For example, the measurement unit 21, training data generation unit 22, machine learning unit 23, and prediction unit 24 can be realized by separate computers (housings).

[0064] Furthermore, all or any part of the processing functions performed by each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware using wired logic.

[0065] (Hardware) Fig. 13 is a diagram showing an example of a hardware configuration. As shown in Fig. 13, an information processing device 10 includes an input device 10a, a network connection device 10b, a storage device 10c, a memory 10d, and a processor 10e. The components shown in Fig. 13 are connected to each other via a bus or the like.

[0066] The input device 10a is a mouse, keyboard, etc., and accepts input of various information. The network connection device 10b is a network interface card, etc., and communicates with other devices. The storage device 10c stores programs and DBs that operate the functions shown in Figure 8.

[0067] The memory 10d has a program load area and a work area. The processor 10e reads a program that executes the same processes as the processing units shown in FIG. 8 from the storage device 10c or the like and loads it into the memory 10d, thereby operating a process that executes each function described in FIG. 8 or the like. For example, this process executes the same functions as the processing units of the information processing device 10. Specifically, the processor 10e reads a program having the same functions as the measurement unit 21, the training data generation unit 22, the machine learning unit 23, the prediction unit 24, or the like from the storage device 10c or the like. Then, the processor 10e executes a process that executes the same processes as the measurement unit 21, the training data generation unit 22, the machine learning unit 23, the prediction unit 24, or the like.

[0068] In this way, the information processing device 10 operates as an information processing device that executes an information processing method by reading and executing a program. The information processing device 10 can also realize functions similar to those of the above-described embodiment by reading the program from a recording medium using a medium reading device and executing the read program. Note that the program in these other embodiments is not limited to being executed by the information processing device 10. For example, the above-described embodiment may also be applied in the same way to cases where another computer or server executes the program, or where these execute the program in cooperation with each other.

[0069] This program may be distributed via a network such as the Internet. Alternatively, this program may be recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, a magneto-optical disk (MO), or a digital versatile disk (DVD), and may be read out from the recording medium and executed by a computer. [Explanation of symbols]

[0070] 10. Information processing equipment 11 Communications Department 12 Storage section 13 Corpus 14 Training Data DB 15 Language Models 20 Control Unit 21 Measuring part 22 Training data generation unit 23 Machine Learning Department 24 Prediction Department

Claims

1. measuring non-functional performance representing performance with respect to requirements other than the functions of each of the plurality of data; performing a machine learning process to train a prediction model that predicts the second part of the data in response to input of the first part of the data by machine learning using divided data obtained by dividing each of the plurality of data into a first part of the data and a second part that is correct answer data as training data; Have the computer execute the process, a loss function including a parameter determined in accordance with the measurement results of the non-functional performance, the parameter indicating a rate at which the non-functional performance is reflected in the prediction model, is used as a loss function in the machine learning processing; A machine learning program characterized by:

2. a loss function including a weight term for setting the parameter and a loss term corresponding to a difference between the correct answer data and a prediction result is used as the loss function in the machine learning processing; The machine learning program according to claim 1 .

3. a loss term based on the appearance probability of a character in each of the plurality of data is used as the loss term of a loss function in the machine learning processing; The machine learning program according to claim 2 .

4. The measuring process includes: measuring the non-functional performance of each of the plurality of programs, excluding functions that define the operation of each of the plurality of programs; The process of executing the machine learning process includes: a machine learning process for training the prediction model that predicts the subsequent part of a program in response to an input of the initial part of the program by machine learning using divided data obtained by dividing each of the plurality of programs into a first part and a subsequent part that is correct answer data as training data; The machine learning program according to claim 2 .

5. The measuring process includes: measuring the non-functional performance indicating an evaluation of direct functions to indirect functions of each of the plurality of document data, excluding functions defining direct uses when using each of the plurality of document data; The process of executing the machine learning process includes: performing a machine learning process to train the prediction model, which predicts the second portion of the document data in response to input of the first portion, by machine learning using training data obtained by dividing each of the plurality of document data into the first portion and the second portion, which is correct answer data; The machine learning program according to claim 2 .

6. measuring non-functional performance representing performance with respect to requirements other than the functions of each of the plurality of data; performing a machine learning process to train a prediction model that predicts the second part of the data in response to input of the first part of the data by machine learning using divided data obtained by dividing each of the plurality of data into a first part of the data and a second part that is correct answer data as training data; The computer executes the processing, a loss function including a parameter determined in accordance with the measurement results of the non-functional performance, the parameter indicating a rate at which the non-functional performance is reflected in the prediction model, is used as a loss function in the machine learning processing; A machine learning method characterized by:

7. measuring non-functional performance representing performance with respect to requirements other than the functions of each of the plurality of data; performing a machine learning process to train a prediction model that predicts the second part of the data in response to input of the first part of the data by machine learning using divided data obtained by dividing each of the plurality of data into a first part of the data and a second part that is correct answer data as training data; A control unit is provided. a loss function including a parameter determined in accordance with the measurement results of the non-functional performance, the parameter indicating a rate at which the non-functional performance is reflected in the prediction model, is used as a loss function in the machine learning processing; 1. An information processing device comprising:

Images