Large model training optimization method, device, computer equipment and storage medium

Abstract

The present application relates to a large model training optimization method, device, computer equipment and storage medium. The method includes: obtaining a training data set; the training data set includes multiple training sample pairs, each training sample pair includes an interface call question sample, multiple interface call answer samples corresponding to the interface call question sample, and standard ranking information of the multiple interface call answer samples; the multiple interface call answer samples are obtained through an initial interface call model; the interface call answer samples are scored by an initial scoring model, and the predicted ranking information of the interface call answer samples is obtained according to the score; the initial scoring model is trained according to the difference between the predicted ranking information and the standard ranking information to obtain a target scoring model; the target interface call model is determined according to the correspondence between the preset scoring model and the preset interface call model and the target scoring model. By adopting the present application, it is possible to output results that are more consistent with the interface call field.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a large-scale model training optimization method, apparatus, computer equipment, and storage medium. Background Art

[0002] With the rapid development of artificial intelligence (AI), AI big models have made significant progress in fields such as natural language processing and image processing. An API (Application Programming Interface) is a set of conventions and tools for interaction and communication between different components of a software system. Developing or using an API requires searching through vast amounts of API documentation for detailed information, such as required API configuration parameters. However, developers or users often only have information related to functional requirements, and cannot find matching API interface information through simple search queries. This allows them to use big models to search for API interface information by entering prompt words.

[0003] However, if the prompt words for querying API interface information are directly input into the large model, the results obtained may not belong to the API field at all, resulting in inaccurate query results. Summary of the Invention

[0004] Based on this, it is necessary to provide a large model training optimization method, device, computer equipment, computer-readable storage medium and computer program product to address the above technical problems, which can improve the accuracy of query results of API interface information.

[0005] In a first aspect, the present application provides a large model training optimization method, comprising:

[0006] Obtain a training data set; the training data set includes multiple training sample pairs, each training sample pair includes an interface call question sample, multiple interface call answer samples corresponding to the interface call question sample, and standard ranking information of the multiple interface call answer samples; the multiple interface call answer samples are obtained through an initial interface call model;

[0007] The interface call answer samples are scored using the initial scoring model, and the predicted ranking information of the interface call answer samples is obtained based on the scores;

[0008] According to the difference between the predicted ranking information and the standard ranking information, the initial scoring model is trained to obtain the target scoring model;

[0009] The target interface call model is determined based on the correspondence between the preset scoring model and the preset interface call model and the target scoring model.

[0010] In a second aspect, the present application provides a large model training optimization device, comprising:

[0011] An acquisition module is used to acquire a training data set; the training data set includes multiple training sample pairs, each training sample pair includes an interface call question sample, multiple interface call answer samples corresponding to the interface call question sample, and standard ranking information of the multiple interface call answer samples; the multiple interface call answer samples are obtained through the initial interface call model;

[0012] A scoring module is used to score the interface call answer samples using an initial scoring model and obtain the predicted ranking information of the interface call answer samples based on the scores;

[0013] A training module is used to train the initial scoring model based on the difference between the predicted ranking information and the standard ranking information to obtain a target scoring model;

[0014] The determination module is used to determine the target interface call model according to the correspondence between the preset scoring model and the preset interface call model and the target scoring model.

[0015] In a third aspect, the present application provides a computer device, which includes a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method when executing the computer program.

[0016] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps in the above-mentioned method when executed by a processor.

[0017] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps in the above method.

[0018] The above-mentioned large model training optimization method, device, computer equipment, computer-readable storage medium and computer program product obtain the trained target scoring model by training the initial scoring model according to the training data set, and determine the target interface call model that has been aligned and trained according to the target scoring model and the correspondence between the preset scoring model and the preset interface call model. It can realize preference alignment training of the initial interface call model, so that the target interface call model can output results that are more in line with the interface call field, thereby improving the accuracy of the output results; in addition, the target scoring model is obtained by training the initial scoring model, and the target interface call model is determined according to the correspondence between the preset scoring model and the preset interface call model. That is, only one model needs to be trained to obtain the model after preference alignment optimization, and there is no need to train multiple models. This reduces the number of model trainings and improves the training efficiency of the interface call model, while reducing the computer video memory usage and improving the computer running speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A diagram illustrating an application environment for a large model training optimization method provided in an embodiment of the present application;

[0020] Figure 2 A flowchart of a large model training optimization method provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of a process for establishing a correspondence between a preset scoring model and a preset interface call model provided in an embodiment of the present application;

[0022] Figure 4 A structural block diagram of a large model training optimization device provided in an embodiment of the present application;

[0023] Figure 5 An internal structure diagram of a computer device provided in an embodiment of the present application;

[0024] Figure 6 An internal structure diagram of another computer device provided in an embodiment of the present application;

[0025] Figure 7 A diagram of the internal structure of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0027] The large model training optimization method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through a communication network. The data storage system can store data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be, but is not limited to, various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0028] like Figure 2 As shown, the embodiment of the present application provides a large model training optimization method, which is applied to Figure 1 The terminal 102 or server 104 in the example is used for explanation. It is understood that the computer device may include at least one of a terminal and a server. The method includes the following steps:

[0029] S202. Obtain a training data set; the training data set includes multiple training sample pairs, each training sample pair includes an interface call question sample, multiple interface call answer samples corresponding to the interface call question sample, and standard ranking information of the multiple interface call answer samples; the multiple interface call answer samples are obtained through the initial interface call model.

[0030] Among them, the interface call question sample refers to a question sample for calling the API interface. The interface call question sample can be data input by the user in the form of text, voice or picture. The interface call question sample can be a sample of inputting the initial interface call model in the expectation of obtaining an interface call answer, which is equivalent to the prompt word inputting the initial interface call model. The interface call question sample can be, for example, "Please find the API interface configuration function that can implement XX function" and so on. The interface call answer sample corresponds to the interface call question sample, that is, the interface call answer sample is an interface call answer generated based on the interface call question sample. It is easy to understand that the format of the interface call answer sample can also be text, voice or picture. The initial interface call model can be any artificial intelligence model, and the artificial intelligence model includes a large model. The initial interface call model can generate answers to the input questions.

[0031] For example, a sample of an interface call question can be input into the initial interface call model to obtain multiple different interface call answer samples, which can then be ranked manually or by a ranking algorithm to obtain standard ranking information for the multiple interface call answer samples. For easy understanding, the standard ranking information can be annotated on the corresponding interface call answer samples.

[0032] It is easy to understand that the more training sample pairs a training dataset includes and the richer the types, the more beneficial it is for model preference alignment training. The final training dataset can be obtained by performing data augmentation on the manually organized training dataset.

[0033] S204: Score the interface call answer samples using the initial scoring model, and obtain predicted ranking information of the interface call answer samples based on the scores.

[0034] The initial scoring model refers to a model that can evaluate answers. It can also be an artificial intelligence model. Typically, the initial scoring model and the initial interface call model are different artificial intelligence models. The initial scoring model can score sample interface call answers based on the evaluation method. The predicted ranking information represents the ranking information obtained by scoring based on the initial scoring model.

[0035] Exemplarily, the interface call answer samples may be scored using an initial scoring model, and multiple interface call answer samples may be ranked according to the scores to obtain predicted ranking information of the interface call answer samples.

[0036] S206: Train the initial scoring model based on the difference between the predicted ranking information and the standard ranking information to obtain a target scoring model.

[0037] The target scoring model refers to the model obtained by performing preference alignment training on the initial scoring model.

[0038] Exemplarily, the initial scoring model can be trained based on the difference between the predicted ranking information and the standard ranking information, so that the scoring model can output predicted ranking information that is approximately consistent with the standard ranking information, thereby obtaining a target scoring model.

[0039] In some embodiments, during the training of the initial scoring model, the initial scoring model is guided to learn the ranking in the standard ranking information, that is, the predicted ranking information corresponding to the interface call answer samples that are ranked higher in the standard ranking information also needs to be greater than the predicted ranking information corresponding to the interface call answer samples that are ranked lower in the standard ranking information, and the probability of judging that it is greater than needs to be higher than a preset threshold in order to obtain the target scoring model.

[0040] In some embodiments, in addition to considering the difference between the predicted ranking information and the standard ranking information, the initial scoring model may also be trained in conjunction with the computer's video memory capacity to obtain a target scoring model. For example, if the difference between the predicted ranking information and the standard ranking information is less than a difference threshold, and the video memory capacity that matches the initial scoring model is less than a capacity threshold, the target scoring model is obtained.

[0041] S208: Determine a target interface call model according to the correspondence between the preset scoring model and the preset interface call model and the target scoring model.

[0042] The target interface call model refers to the interface call model obtained by performing preference alignment optimization on the initial interface call model. It should be noted that the preset scoring model is a pre-selected artificial intelligence model that can score input answers. The preset interface call model is a pre-selected artificial intelligence model that can output interface call information.

[0043] For example, a correspondence between a preset scoring model and a preset interface call model can be established based on an initial alignment optimization objective function. The initial alignment optimization objective function is determined by the preset interface call model, a preset reference model, and a preset scoring model. For example, the initial alignment optimization objective function can be determined based on the difference between the preset interface call model and the preset reference model, and the score of the interface call answer generated by the preset interface call model by the preset scoring model. The optimization goal is to minimize the difference between the preset interface call model and the preset reference model, while maximizing the score of the interface call answer generated by the preset scoring model by the preset scoring model. For example, the objective function of preference alignment optimization using the PPO (Proximal Policy Optimization) algorithm can be used as the initial alignment optimization objective function. The initial alignment optimization objective function can be transformed to remove or replace the preset reference model, thereby obtaining a correspondence between the preset scoring model and the preset interface call model.

[0044] It can be seen that in the embodiment of the present application, the initial scoring model is trained according to the training data set to obtain the trained target scoring model, and the target interface call model that has been aligned and trained is determined according to the target scoring model and the correspondence between the preset scoring model and the preset interface call model, which can indirectly realize the preference alignment training of the initial interface call model, so that the interface call model can output results that are more in line with the interface call field, thereby improving the accuracy of the output results; in addition, the target scoring model is obtained by training the initial scoring model, and the target interface call model is determined according to the correspondence between the preset scoring model and the preset interface call model, that is, only one model needs to be trained to obtain the preference alignment optimized model, and there is no need to train multiple models, which reduces the number of model training and improves the training efficiency of the interface call model, while reducing the computer video memory usage and improving the computer running speed.

[0045] In some embodiments, as Figure 3 As shown, before determining the target interface call model based on the correspondence between the preset scoring model and the preset interface call model and the target scoring model, the above method further includes:

[0046] S302 , obtaining an initial alignment optimization objective function; wherein the initial alignment optimization objective function is determined by a preset interface calling model, a preset reference model, and a preset scoring model.

[0047] Among them, the initial alignment optimization objective function is an optimization objective function for performing preference alignment optimization on the preset interface call model to be trained. During the optimization training of the preset interface call model, some auxiliary models can be used to participate in the preference alignment optimization of the preset interface call model to be trained. The auxiliary models include a preset reference model and a preset scoring model. Among them, the preset interface call model and the preset reference model can be any different artificial intelligence models. The preset scoring model is a model that can score the results output by the preset interface call model. Exemplarily, the KL divergence (Kullback-Leibler Divergence) between the preset interface call model and the preset reference model can be included in the initial alignment optimization objective function. This can prevent the output results of the preset interface call model from deviating too far from the output results of the preset reference model during the training of the preset interface call model, resulting in non-convergence of the model training.

[0048] In some embodiments, the initial alignment optimization objective function includes the KL divergence between the output result of the preset interface call model and the output result of the preset reference model, and the output result of the preset scoring model, wherein the preset interface call model is the model to be trained, the preset scoring model is the trained model, and the preset reference model is the model used as a reference for training the preset interface call model. For example, the result of the initial alignment optimization objective function is the difference between the KL divergence between the output result of the preset scoring model and the output result of the preset interface call model and the output result of the preset reference model. The initial alignment optimization objective function can be shown as the following formula (1).

[0049] Formula (1)

[0050] in, Indicates the preset interface call model; Represents the preset scoring model; Represents the preset reference model; x represents the input prompt word, that is, the interface call question; D represents the set corresponding to x; y represents the interface call answer corresponding to the output of input x. represents the hyperparameter, Represents the KL divergence, and E represents the optimization objective. The first half of the initial alignment optimization objective function is equivalent to maximizing the output value of the preset scoring model, and the second half is to minimize the difference between the preset interface call model and the preset reference model. In other words, when the output value of the preset scoring model is greater than the preset scoring threshold, and the difference between the preset interface call model and the preset reference model is less than the difference threshold, the corresponding optimization objective is achieved and the optimized interface call model is obtained. It is easy to understand that the initial alignment optimization objective function obtained by deforming formula (1) should also be within the protection scope corresponding to the embodiment of the present application. For example, formula (2) obtained by deforming formula (1) is shown below.

[0051] Formula (2)

[0052] The maximization problem in formula (2) can be transformed into a minimization problem, then formula (2) can be transformed into the following formula (3)

[0053] Formula (3)

[0054] That is, the initial alignment optimization objective function can be as shown in Formula (1), Formula (2) or Formula (3).

[0055] S304 , substituting the obtained partial function into the initial alignment optimization objective function to obtain the objective function to be optimized; wherein the partial function is a function with a preset scoring model and a preset reference model as independent variables.

[0056] In actual application scenarios, the role of the partial function is to remove the preset scoring model from the optimization process. That is, during the transformation process, the parameters of the preset scoring model will not be changed accordingly.

[0057] For example, the partial function can be expressed by the following formula (4).

[0058] Formula (4)

[0059] For example, in formula (4) It means that under the premise of a certain prompt word X, all the y that may be generated by the preset reference model, so , and the partial function expressed by formula (4) is a function of x, and If it is irrelevant, substitute the partial function shown in formula (4) into the initial alignment optimization objective function shown in formula (3), and the objective function to be optimized is obtained as shown in the following formula (5).

[0060] Formula (5)

[0061] S306: Determine a display solution of a preset interface call model according to the objective function to be optimized.

[0062] An explicit solution refers to a solution to a mathematical equation or problem that can be directly expressed as an explicit function of a known function. This typically involves a functional relationship where the dependent variable can be clearly expressed as an independent variable. Determining the explicit solution of the interface call model based on the objective function to be optimized is equivalent to directly calculating the parameters of the preset interface call model based on the preset reference model and the preset scoring model.

[0063] For example, according to the objective function to be optimized corresponding to formula (5), the display solution of the preset interface call model is determined as shown in the following formula (6). In the process of solving the display solution, the expectation in formula (5) should be minimized, because and The optimization is irrelevant. In the gradient descent algorithm, the model update will lead to the gradient update regardless of whether it is positive or negative. Therefore, the minimum update is 0, that is, The value of is 0, log1=0, which means we get the value shown in formula (6) Display solution of .

[0064] Formula (6)

[0065] S308: Determine the correspondence between the preset scoring model and the preset interface calling model according to the display solution of the preset interface calling model.

[0066] After obtaining the display solution of the preset interface call model, it is equivalent to obtaining the correspondence between the preset interface call model, the preset scoring model and the preset reference model. Then, after removing the preset reference model, the correspondence between the preset interface call model and the preset scoring model can be obtained.

[0067] Exemplarily, the maximum likelihood estimation result corresponding to the preset interface call model can be calculated, and the preset reference model can be replaced by the maximum likelihood estimation result to obtain the corresponding relationship between the preset scoring model and the preset interface call model.

[0068] It can be seen that in this embodiment, the display solution of the preset interface call model is obtained by transformation processing through partial functions and initial alignment optimization objective functions. According to the display solution of the preset interface call model, the correspondence between the preset scoring model and the preset interface call model is determined, which can ensure that the output of the preset interface call model to be trained is a valid probability distribution and can improve the accuracy of the correspondence between the preset scoring model and the preset interface call model.

[0069] In some embodiments, determining the correspondence between the preset scoring model and the preset interface call model according to the display solution of the preset interface call model includes:

[0070] Convert the displayed solution of the preset interface call model into an expression with the preset scoring model as the dependent variable;

[0071] Generate the maximum likelihood estimation result corresponding to the preset interface calling model;

[0072] The preset reference model in the expression is replaced with the maximum likelihood estimation result to obtain the corresponding relationship between the preset scoring model and the preset interface call model.

[0073] The display solution is an expression with the preset interface call model as the dependent variable. Transforming the display solution yields an expression with the preset scoring model as the dependent variable. For example, transforming Formula (6) yields the expression with the preset interface call model as the dependent variable, as shown in Formula (7).

[0074] Formula (7)

[0075] From formula (7), we can see that this is equivalent to substituting the preset interface call model to be optimized into the optimization target of the preset scoring model. In other words, the training steps of the preset interface call model can be integrated into the training of the preset scoring model. The maximum likelihood estimation result corresponding to the preset interface call model is generated as shown in the following formula (8).

[0076] Formula (8)

[0077] During the training process, suppose an interface call question is input into the preset interface call model to be trained, and two interface call answers are generated. The better one is manually determined as y. w , the worse one is y l , in the optimization process, the ranking corresponding to the preset scoring model is , but it does not necessarily mean that the likelihood ranking satisfies Therefore, the ranking of the preset scoring model can be replaced by the likelihood ranking, that is, the preset reference model is replaced by the maximum likelihood estimation result, and the following formula (9) is obtained.

[0078] Formula (9)

[0079] Formula (9) represents the correspondence between the preset scoring model and the preset interface calling model.

[0080] It can be seen that in this embodiment, by replacing the preset reference model with the maximum likelihood estimation result corresponding to the preset interface call model in the expression with the preset scoring model as the dependent variable, the correspondence between the preset scoring model and the preset interface model is obtained, which can achieve consistency between the likelihood ranking and the manual ranking results, thereby improving the accuracy of the correspondence between the preset scoring model and the preset interface model.

[0081] In some embodiments, the predicted ranking information includes a first predicted ranking and a second predicted ranking; and training an initial scoring model based on a difference between the predicted ranking information and the standard ranking information to obtain a target scoring model includes:

[0082] Obtaining a first predicted ranking obtained by predicting the first interface call answer sample using the initial scoring model, and a second predicted ranking obtained by predicting the second interface call answer sample; wherein the ranking of the first interface call answer sample in the standard ranking information is higher than the ranking of the second interface call answer sample in the standard ranking information;

[0083] determining a probability that the first predicted rank is greater than the second predicted rank;

[0084] When the probability is greater than a preset probability threshold, the target scoring model is determined.

[0085] The initial scoring model can be an artificial intelligence model that scores interface call answer samples based on the frequency of occurrence of target terms in the interface call answer samples, which are used to represent the standard answer. Alternatively, the initial scoring model can calculate the keyword overlap rate between the interface call answer samples and the standard answer and score the interface call answer samples based on this keyword overlap rate. Alternatively, the initial scoring model can calculate the Levenshtein distance or cosine similarity between the interface call answer samples and the standard answer and score the interface call answer samples based on the Levenshtein distance or cosine similarity. Alternatively, the initial scoring model can collect data such as user browsing frequency, click frequency, and answer accuracy feedback on different interface call answer samples and score the interface call answer samples based on this data. The predicted ranking information can be understood as the predicted ranking of the interface call answer samples.

[0086] Exemplarily, the initial scoring model scores the first interface call answer sample to obtain a first answer score, and scores the second interface call answer sample to obtain a second score, and ranks the first answer score and the second answer score to obtain a first predicted ranking corresponding to the first answer score and a second predicted ranking corresponding to the second answer score. The ranking of the first interface call answer sample in the standard ranking information is before the ranking of the second interface call answer sample in the standard ranking information. Therefore, during the training process, it is also necessary to strive to make the first predicted ranking obtained by the initial scoring model greater than the second predicted ranking. Therefore, the probability that the first predicted ranking is greater than the second predicted ranking is calculated. When the probability is greater than a preset probability threshold, the target scoring model is obtained. Among them, the preset probability threshold can be set according to the actual application scenario. For example, the preset probability threshold can be 0.95, 0.96 or 0.99, etc.

[0087] In actual application scenarios, the same interface call question sample may correspond to more than or equal to two interface call answer samples. The probability calculation of the predicted ranking can be performed for different interface call answer samples of the same interface call question sample, or the probability calculation of the predicted ranking can be performed for interface call answer samples corresponding to multiple interface call question samples. Specifically, the probability that the first predicted ranking is greater than the second predicted ranking in the predicted rankings corresponding to different interface call answer samples of the same interface call question sample can be counted. Alternatively, the probability that the first predicted ranking is greater than the second predicted ranking in the predicted rankings of interface call answer samples corresponding to different interface call question samples can also be counted. Until the statistical probability is greater than the preset probability threshold, the target scoring model is obtained.

[0088] In one example, the Bradley-Terry model (a statistical model of sports games) can be used for modeling, and formula (10) is obtained.

[0089] Formula (10)

[0090] Among them, formula (10) can calculate y w Greater than y l probability.

[0091] In order to make y w As much as possible greater than y l , for the entire training dataset , N represents the number of training samples. The optimization objective of the scoring model can be expressed as follows (11).

[0092] Formula (11)

[0093] Will The specific form of , that is, formula (10) is substituted into formula (11), and the optimization objective of the scoring model can be obtained as shown in the following formula (12).

[0094] Formula (12)

[0095] Among them, the optimization goal of the scoring model is usually described as a loss problem. In formula (12), when the scoring model is used to calculate y w The ranking is greater than y l When the ranking of is maximized, the corresponding loss is minimized.

[0096] It can be seen that in this embodiment, by determining the probability that the predicted ranking obtained by the initial scoring model for the interface call answer sample that ranks higher in the standard ranking information is greater than the predicted ranking obtained by predicting the interface call answer sample that ranks lower in the standard ranking information, when the probability is greater than the preset probability threshold, it means that the predicted ranking obtained by the initial scoring model for the interface call answer sample is relatively consistent with the ranking in the standard ranking information, that is, the ranking of the interface call answer sample by the initial scoring model is consistent with the standard ranking, thereby obtaining a more accurate target scoring model.

[0097] In some embodiments, the above method further comprises:

[0098] Obtaining a first score obtained by scoring the first interface call answer sample using the initial scoring model, and a second score obtained by scoring the second interface call answer sample;

[0099] Determine the training loss based on the difference between the first score and the second score and the score difference term;

[0100] The initial scoring model is trained according to the training loss to obtain the target scoring model.

[0101] The score difference term is used to reduce the possibility of overfitting in model training. If the difference between the first and second scores is too large, the model may be biased towards the answer sample of the first interface call corresponding to the first score; conversely, if the difference between the first and second scores is too small, the model may be biased towards the answer sample of the second interface call corresponding to the second score, which will affect the generalization performance of the model. Therefore, the score difference term is used to limit overfitting in model training. The score difference term can be a constant greater than zero, usually in the range of (0,1]. For example, the score difference term can be 0.2 or 0.3.

[0102] Exemplarily, the initial scoring model is trained according to the training loss, that is, the parameters of the initial scoring model are adjusted so that the training loss becomes smaller, until the training loss is less than the loss threshold, and the target scoring model is obtained.

[0103] In some embodiments, the specific form of the initial scoring model (i.e., replacing the preset scoring model with the initial scoring model in formula (9)) can be substituted into formula (12), thereby converting the optimization goal of the initial scoring model into the optimization goal of the initial interface call model, as shown in the following formula (13).

[0104] Formula (13)

[0105] Since the difference between the two classes affects the generalization ability of the model classifier, in the preference alignment training, the two classes are the winning or losing responses of a single input, and the score difference term is introduced in the optimization objective of the initial interface call model. , the final optimization target of the initial interface call model can be obtained as shown in the following formula (14).

[0106] Formula (14)

[0107] It can be seen that in this embodiment, by determining the training loss of the initial scoring model based on the difference between the score of the first interface call answer sample that ranks higher in the standard ranking information and the score of the second interface call answer sample that ranks lower in the standard ranking information according to the initial scoring model, as well as the score difference item, the overfitting situation in the model training process can be reduced and the generalization ability of the target scoring model can be improved.

[0108] In some embodiments, the initial scoring model is trained based on the difference between the predicted ranking information and the standard ranking information to obtain a target scoring model, including:

[0109] Determine the memory capacity that matches the initial scoring model;

[0110] Determine the training conditions based on the difference between the predicted ranking information and the standard ranking information and the memory capacity that matches the initial scoring model;

[0111] The initial scoring model is trained, and when the training conditions are met, the target scoring model is obtained.

[0112] GPU memory capacity refers to the amount of video memory on a computer's graphics processing unit (GPU). The size of the GPU determines its ability to temporarily store data. GPU memory capacity can be, for example, 128MB, 256MB, 512MB, 1024MB, 2GB, 4GB, 8GB, or 1TB. The GPU memory capacity that matches the initial scoring model can be understood as the amount of memory occupied by the initial scoring model during its execution.

[0113] Exemplarily, the training conditions include a difference condition and a video memory condition. During the training process of the initial scoring model, the video memory capacity that matches the initial scoring model can be calculated in real time, and the target scoring model is obtained when the difference between the predicted ranking information and the standard ranking information satisfies the difference condition, and the video memory capacity that matches the initial scoring model satisfies the video memory condition. The difference condition may be, for example, less than or equal to a difference threshold, and the video memory condition may be, for example, less than or equal to a preset video memory capacity. The difference threshold and the preset video memory capacity may be set according to the actual application scenario, for example, the difference threshold is 0.2, 0.1 or 0.01, etc. The preset video memory capacity may be less than or equal to the maximum video memory capacity of the graphics card in the computer, for example, the preset video memory capacity is 50%, 70% or 80% of the computer graphics card capacity, etc.

[0114] It can be seen that in this embodiment, the training conditions of the initial scoring model are determined based on the difference between the predicted ranking information and the standard ranking information and the video memory capacity that matches the initial scoring model. When the training conditions are met, the target scoring model is obtained. It is possible to use the video memory capacity occupied by the initial scoring model as a training condition to participate in the training of the initial scoring model, so that the video memory capacity occupied by the trained model meets the preset conditions, thereby achieving effective control of the video memory capacity occupied by the training model.

[0115] In an exemplary application scenario, an interface call question sample x is input into the initial interface call model, and two interface call answer samples y are obtained. w and y l It is easy to understand that in other application scenarios, three or more interface call answer samples may be obtained. This example uses two as an example.w and y l Ranking is performed and the standard ranking information is obtained as y w Greater than y l The initial scoring model is used to call the answer sample y of the two interfaces respectively. w and y l Score and get y w The corresponding first score, and y l The corresponding second score can be obtained based on the first score and the second score. w and y l The predicted ranking information is obtained by training the initial scoring model based on the difference between the predicted ranking information and the standard ranking information until the difference between the predicted ranking information and the standard ranking information is less than a difference threshold, thereby obtaining a target scoring model. The target interface call model is then determined based on the correspondence between the preset scoring model and the preset interface call model, as well as the target scoring model. Alternatively, the initial interface call model can be trained based on the difference between the predicted ranking information and the standard ranking information until the difference between the predicted ranking information and the standard ranking information is less than a difference threshold, thereby obtaining a target interface call model.

[0116] In some embodiments, during the alignment training of the initial interface call model using the PPO algorithm, the coordinated operation of multiple models is required, resulting in problems such as large video memory usage, a large number of training hyperparameters, and unstable training during the training process. The optimization target of the PPO algorithm can be used as the initial alignment optimization objective function. By transforming the optimization target of the PPO algorithm, a partial function with a preset scoring model and a preset reference model as dependent variables is configured, and the maximum likelihood estimation result corresponding to the preset interface call model is determined. The optimization target of the PPO algorithm is transformed according to the partial function and the maximum likelihood estimation result to obtain the corresponding relationship between the preset scoring model and the preset interface call model, that is, the relevant parameters of the preset reference model are eliminated. Thus, only the initial scoring model needs to be trained to obtain the target scoring model. According to the corresponding relationship between the preset scoring model and the preset interface call model, the target interface call model corresponding to the target scoring model is obtained. This can reduce the types of training data, the number of models trained and the number of hyperparameters, reduce the parameter adjustment cost, and reduce the video memory requirements of multi-model collaboration in PPO, so that the model training can be completed with smaller video memory resources, thereby improving the computer operation speed.

[0117] When the target interface call model is actually used, the interface call question is input into the target interface call model to obtain an interface call answer that is more consistent with the API field. The interface call answer may include information such as the configuration function, class, return type, and parameters corresponding to the API interface to be called.

[0118] It should be understood that, although the steps in the flowcharts of the above-mentioned embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above-mentioned embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0119] Based on the same inventive concept, the present application also provides a large-model training optimization device. The implementation solution provided by this device is similar to the implementation solution described in the above method. Therefore, the specific limitations of one or more large-model training optimization device embodiments provided below can be found in the above-mentioned limitations of the large-model training optimization method and will not be repeated here.

[0120] like Figure 4 As shown, the embodiment of the present application provides a large model training optimization device 400, including:

[0121] Acquisition module 402 is used to acquire a training data set; the training data set includes multiple training sample pairs, each training sample pair includes an interface call question sample, multiple interface call answer samples corresponding to the interface call question sample, and standard ranking information of the multiple interface call answer samples; the multiple interface call answer samples are obtained by the initial interface call model;

[0122] Scoring module 404, configured to score the interface call answer samples using an initial scoring model, and obtain predicted ranking information of the interface call answer samples based on the scores;

[0123] A training module 406 is configured to train the initial scoring model based on the difference between the predicted ranking information and the standard ranking information to obtain a target scoring model;

[0124] The determination module 408 is configured to determine a target interface call model according to the correspondence between the preset scoring model and the preset interface call model and the target scoring model.

[0125] In some embodiments, the above-mentioned device also includes a construction module, which is used to: obtain an initial alignment optimization objective function; the initial alignment optimization objective function is determined by a preset interface call model, a preset reference model and a preset scoring model; substitute the obtained partial function into the initial alignment optimization objective function to obtain the objective function to be optimized; the partial function is a function with a preset scoring model and a preset reference model as independent variables; according to the objective function to be optimized, determine the display solution of the preset interface call model; according to the display solution of the preset interface call model, determine the correspondence between the preset scoring model and the preset interface call model.

[0126] In some embodiments, in determining the correspondence between the preset scoring model and the preset interface call model according to the display solution of the preset interface call model, the construction module is specifically configured to:

[0127] Convert the displayed solution of the preset interface call model into an expression with the preset scoring model as the dependent variable;

[0128] Generate the maximum likelihood estimation result corresponding to the preset interface calling model;

[0129] The preset reference model in the expression is replaced with the maximum likelihood estimation result to obtain the corresponding relationship between the preset scoring model and the preset interface call model.

[0130] In some embodiments, the predicted ranking information includes a first predicted ranking and a second predicted ranking; in training the initial scoring model based on the difference between the predicted ranking information and the standard ranking information to obtain a target scoring model, the training module 406 is specifically configured to:

[0131] Obtaining a first predicted ranking obtained by predicting the first interface call answer sample using the initial scoring model, and a second predicted ranking obtained by predicting the second interface call answer sample; the ranking of the first interface call answer sample in the standard ranking information is higher than the ranking of the second interface call answer sample in the standard ranking information;

[0132] determining a probability that the first predicted rank is greater than the second predicted rank;

[0133] When the probability is greater than a preset probability threshold, the target scoring model is determined.

[0134] In some embodiments, the training module 406 is further configured to:

[0135] A first score obtained by scoring the first interface call answer sample through the initial scoring model and a second score obtained by scoring the second interface call answer sample are obtained respectively; a training loss is determined based on the difference between the first score and the second score and the score difference item; the initial scoring model is trained based on the training loss to obtain a target scoring model.

[0136] In some embodiments, in training the initial scoring model based on the difference between the predicted ranking information and the standard ranking information to obtain the target scoring model, the training module 406 is specifically configured to:

[0137] Determine the memory capacity that matches the initial scoring model;

[0138] Determine the training conditions based on the difference between the predicted ranking information and the standard ranking information and the memory capacity that matches the initial scoring model;

[0139] The initial scoring model is trained, and when the training conditions are met, the target scoring model is obtained.

[0140] Each module in the large-scale model training and optimization device described above can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0141] In some embodiments, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store data related to large model training optimization. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in the above-mentioned large model training optimization method are implemented.

[0142] In some embodiments, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals via wired or wireless means, and the wireless means can be achieved via Wi-Fi, mobile cellular networks, NFC (near-field communication), or other technologies. When executed by the processor, the computer program implements the steps of the large model training optimization method described above. The display unit of the computer device is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen; the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0143] Those skilled in the art will understand that Figure 5 or Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0144] In some embodiments, a computer device is provided. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps in the above method embodiments are implemented.

[0145] In some embodiments, as Figure 7 The figure shows an internal structure diagram of a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0146] In some embodiments, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0147] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0148] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the above-described method embodiments. Any reference to a memory, database, or other medium used in the embodiments provided herein may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processors (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0149] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0150] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A large model training optimization method, characterized in that: include: Get the training dataset; The training data set includes a plurality of training sample pairs, each of which includes an interface call question sample, a plurality of interface call answer samples corresponding to the interface call question sample, and standard ranking information of the plurality of interface call answer samples; the plurality of interface call answer samples are obtained through an initial interface call model; Scoring the interface call answer samples using an initial scoring model, and obtaining predicted ranking information of the interface call answer samples based on the scores; Training the initial scoring model according to the difference between the predicted ranking information and the standard ranking information to obtain a target scoring model; The target interface call model is determined according to the correspondence between the preset scoring model and the preset interface call model and the target scoring model.

2. The method according to claim 1, characterized in that Before determining the target interface call model based on the correspondence between the preset scoring model and the preset interface call model and the target scoring model, the method further includes: Obtaining an initial alignment optimization objective function; the initial alignment optimization objective function is determined by a preset interface calling model, a preset reference model, and a preset scoring model; Substituting the obtained partial function into the initial alignment optimization objective function to obtain the objective function to be optimized; the partial function is a function with the preset scoring model and the preset reference model as independent variables; Determining a display solution of the preset interface call model according to the objective function to be optimized; According to the display solution of the preset interface call model, a corresponding relationship between the preset scoring model and the preset interface call model is determined.

3. The method according to claim 2, characterized in that The determining, according to the display solution of the preset interface call model, the corresponding relationship between the preset scoring model and the preset interface call model includes: Converting the display solution of the preset interface call model into an expression with the preset scoring model as a dependent variable; Generate a maximum likelihood estimation result corresponding to the preset interface call model; The preset reference model in the expression is replaced by the maximum likelihood estimation result to obtain a corresponding relationship between the preset scoring model and the preset interface call model.

4. The method according to claim 1, wherein The predicted ranking information includes a first predicted ranking and a second predicted ranking; and the training of the initial scoring model based on the difference between the predicted ranking information and the standard ranking information to obtain a target scoring model includes: Obtaining respectively the first predicted ranking obtained by predicting the first interface call answer sample using the initial scoring model, and the second predicted ranking obtained by predicting the second interface call answer sample; the ranking of the first interface call answer sample in the standard ranking information is higher than the ranking of the second interface call answer sample in the standard ranking information; determining a probability that the first predicted rank is greater than the second predicted rank; When the probability is greater than a preset probability threshold, a target scoring model is determined.

5. The method according to claim 4, characterized in that The method further comprises: Obtaining a first score obtained by scoring the first interface call answer sample using the initial scoring model, and a second score obtained by scoring the second interface call answer sample; Determining a training loss based on a difference between the first score and the second score and a score difference term; The initial scoring model is trained according to the training loss to obtain a target scoring model.

6. The method according to claim 1, wherein The training of the initial scoring model according to the difference between the predicted ranking information and the standard ranking information to obtain a target scoring model includes: Determining a video memory capacity that matches the initial scoring model; determining a training condition based on a difference between the predicted ranking information and the standard ranking information and a video memory capacity that matches the initial scoring model; The initial scoring model is trained to obtain a target scoring model when the training conditions are met.

7. A large model training optimization device, characterized in that: include: Acquisition module, used to obtain training data sets; The training data set includes a plurality of training sample pairs, each of which includes an interface call question sample, a plurality of interface call answer samples corresponding to the interface call question sample, and standard ranking information of the plurality of interface call answer samples; the plurality of interface call answer samples are obtained through an initial interface call model; A scoring module, configured to score the interface call answer samples using an initial scoring model, and obtain predicted ranking information of the interface call answer samples based on the scores; A training module, configured to train the initial scoring model according to the difference between the predicted ranking information and the standard ranking information to obtain a target scoring model; The determination module is configured to determine a target interface call model based on a correspondence between a preset scoring model and a preset interface call model and the target scoring model.

8. The device according to claim 7, characterized in that The apparatus further includes a construction module configured to: obtain an initial alignment optimization objective function; the initial alignment optimization objective function is determined by a preset interface call model, a preset reference model, and a preset scoring model; substitute the obtained partial function into the initial alignment optimization objective function to obtain an objective function to be optimized; the partial function is a function with the preset scoring model and the preset reference model as independent variables; and determine a display solution of the preset interface call model based on the objective function to be optimized; According to the display solution of the preset interface call model, a corresponding relationship between the preset scoring model and the preset interface call model is determined.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images