Model training method and device, equipment, medium and program product

Abstract

The embodiment of the invention provides a model training method and device, equipment, a medium and a program product, and the method comprises the steps: obtaining a model training request in a model training process, and determining a to-be-trained model in a model library according to a keyword in the model training request, generating test data corresponding to the model type of the to-be-trained model through the large language model, inputting the test data into the to-be-trained model for test processing to obtain a test record, generating a test result according to the test record and the test label, and determining a data type of a target index of which the index is smaller than a threshold value in the test result, the training data corresponding to the data type is generated through the large language model for model training, and the model to be trained is trained through the large language model, so that the reliability of model training and the accuracy of a model output result are improved.

Description

Technical Field

[0001] This document relates to the field of data processing technology, and in particular to a model training method, device, equipment, medium and program product. Background Art

[0002] With the continuous development of Internet technology and the increasing computing needs, more and more users have begun to use pre-trained models for repeated computing. Pre-trained models can improve data processing efficiency and calculation accuracy. However, as more and more users use pre-trained models for data processing, problems such as low calculation accuracy and high model training costs have also arisen. In the prior art, model training is usually carried out by independently constructing a training set. However, there is a problem that the data in the training set is unreliable, resulting in low reliability of model training. How to train a more reliable model has become an issue that all parties need to pay attention to. Summary of the invention

[0003] The purpose of one embodiment of the present specification is to provide a model training method, apparatus, device, medium and program product to solve the problem of inaccurate model output results caused by unreliable models during model training.

[0004] To solve the above technical problems, an embodiment of this specification is implemented as follows: In a first aspect, an embodiment of the present specification provides a model training method, comprising: Obtain a model training request, and determine a model to be trained in a model library according to keywords in the model training request; Generate test data corresponding to the model type of the model to be trained by using a large language model, and input the test data into the model to be trained for test processing to obtain a test record; Generate a test result according to the test record and the label of the test data, and determine the data type corresponding to the target indicator whose indicator in the test result is less than the indicator threshold; The training data of the data type of the model to be trained is generated by the large language model, and the model to be trained is trained based on the training data.

[0005] In a second aspect, another embodiment of the present specification provides a model training device, comprising: A module for determining a model to be trained, used to obtain a model training request and determine a model to be trained in a model library according to keywords in the model training request; A model testing module, used to generate test data corresponding to the model type of the model to be trained through a large language model, and input the test data into the model to be trained for testing and obtaining a test record; A data type determination module, configured to generate a test result according to the test record and the label of the test data, and determine a data type corresponding to a target indicator in the test result whose indicator is less than an indicator threshold; A model training module is used to generate training data of the data type of the model to be trained through the large language model, and train the model to be trained based on the training data.

[0006] In the third aspect, another embodiment of the present specification provides a model training device, comprising: a memory, a processor, and computer executable instructions stored on the memory and executable on the processor, wherein the computer executable instructions, when executed by the processor, implement the steps of the model training method as described in the first aspect above.

[0007] In a fourth aspect, another embodiment of the present specification provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps of the model training method described in the first aspect above are implemented.

[0008] In a fifth aspect, another embodiment of the present specification provides a computer program product, wherein the computer program product includes a model training program, and when the model training program is executed by a processor, the steps of the model training method described in the first aspect above are implemented.

[0009] The model training method provided in this embodiment first obtains a model training request through the interaction between the user and the intelligent agent during the model training process, determines the model to be trained in the model library according to the keywords in the model training request, generates test data corresponding to the model type of the model to be trained through the large language model, generates targeted test data through the large language model, improves the reliability of the model test result, inputs the test data into the model to be trained for test processing to obtain a test record, generates a test result according to the test record and the label of the test data, determines the test result of the model by defining various indicators, generates training data for the model in a more targeted manner, improves the availability of model training, determines the data type corresponding to the target indicator whose indicator in the test result is less than the indicator threshold, finally generates training data of the data type of the model to be trained through the large language model, generates targeted training data through the large language model, trains the model to be trained based on the training data, and performs targeted model training reinforcement for the data type, thereby improving the reliability of model training without increasing the training cost and improving the availability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate one or more embodiments of the present specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present specification. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. Figure 1 A processing flow chart of a model training method provided for one or more embodiments of this specification; Figure 2 A processing flow chart of a model training method applied to a model training scenario provided in one or more embodiments of this specification; Figure 3 A schematic diagram of a model training method processing flow provided for one or more embodiments of this specification; Figure 4 A schematic diagram of a model training device provided for one or more embodiments of this specification; Figure 5 A schematic diagram of the structure of a model training device provided for one or more embodiments of this specification. DETAILED DESCRIPTION

[0011] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will be combined with the drawings in one or more embodiments of this specification to clearly and completely describe the technical solutions in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.

[0012] This specification provides an embodiment of a model training method: Reference Figure 1 , which shows a processing flow chart of a model training method provided in this embodiment, referring to Figure 2 , which shows a processing flow chart of a model training method provided in this embodiment and applied to a model training scenario.

[0013] Reference Figure 1 The model training method provided in this embodiment specifically includes the following steps S102 to S108.

[0014] Step S102: Obtain a model training request, and determine a model to be trained in a model library according to keywords in the model training request.

[0015] In specific implementation, an external input model training request is obtained, keywords of the model in the model training request are extracted, and the model to be trained corresponding to the keyword is determined in the model library based on the keyword. Specifically, the model training request can be generated through the content of the interaction input between the user and the agent, and the keyword can be the keyword of the model type of the interaction information input by the user. Optionally, the keyword includes: type keyword and / or name keyword of the model to be trained.

[0016] In this embodiment, the model to be trained can be a model with fewer parameters than the large model, and can be a model that performs a specific task, such as a classification model, a generation model and / or a prediction model, and can specifically be an image classification model, a text classification model and / or a text generation model. In addition, the model to be trained can also be a trained model, and the trained model to be trained can be trained again after performance testing to obtain a more reliable and more usable model.

[0017] In practical applications, in order to improve the efficiency of model testing, optimization and / or training, an agent may be configured for user interaction. The user may interact with the agent through text, voice and / or gesture to input model training information, and generate a model training request based on the model training information. In an optional implementation manner provided in this embodiment, before obtaining the model training request, the following is also included: The model training information of the model to be trained that is interactively input by the user and the agent is obtained, and keywords in the model training information are extracted to generate a model training request.

[0018] Specifically, model keywords and / or task keywords in the model training information can be extracted to generate a model training request containing the model keywords and / or task keywords, so as to determine the model to be trained according to the model keywords and determine the task to be processed according to the task keywords.

[0019] For example, if the user inputs "Please optimize model x", the model to be trained is determined to be model x, and the subtasks are converted into "1. Performance test 2. Performance evaluation 3. Optimization training"; For another example, if the user inputs: "Please test the classification model", the model to be trained is determined to be a classification model, and the converted subtask is: "performance test".

[0020] It should be noted that the user can perform multiple optimization trainings for the model to be trained. The specific process of multiple optimization trainings can be explained by taking the process of a single optimization training in this embodiment as an example. The specific operations can be performed with reference to the operations of the single optimization training in this embodiment, which will not be repeated here.

[0021] During the specific execution process, the model to be trained can be placed in the model library and the keywords of the model can be configured. The user can interact with the agent to determine the model to be processed and the operation to be performed on the model. The accuracy of determining the model to be trained is improved by configuring keywords. In an optional implementation manner provided by this embodiment, the model to be trained is determined according to the keywords, including: Extracting the name keyword and / or type keyword of the model to be trained in the model training request, and calling a query interface according to the type keyword to perform a model query in the model library; The target model returned by the interface call is obtained, and the target model is used as the model to be trained.

[0022] Specifically, the model query processing can be performed through the agent tool, the name keyword and / or type keyword can be used as input data to call the agent tool, the model query can be performed through the agent tool, and the target model returned by the agent tool can be used as the model to be trained. For example, the model query processing can be performed through the model API calling tool and / or database tool.

[0023] Step S104: Generate test data corresponding to the model type of the model to be trained through the large language model, and input the test data into the model to be trained for test processing to obtain a test record.

[0024] In a specific implementation, the name keywords and / or type keywords of the model to be trained are input into the large language model, so that the large language model generates test data corresponding to the model type of the model to be trained, and the test data is input into the model to be trained for test processing to obtain the test record of the model to be trained. Specifically, the name keywords and / or type keywords of the model to be trained and the prompt words corresponding to the generation of test data can also be input into the large language model, so that the large language model generates test data.

[0025] During the specific execution process, the name keywords and / or type keywords of the model to be trained can be input into the large language model. The large language model automatically generates test data and defines relevant evaluation indicators to determine the test results based on the test records and evaluation indicators, that is, the performance of the model to be trained. The reliability and accuracy of model training and / or model testing are improved through the large language model.

[0026] Specifically, the model to be trained may be a classification model. When the model to be trained is a multi-classification model, test data of each classification may be generated, and the model to be trained may be tested by the test data of each classification to determine the classification accuracy of the model to be trained. In an optional implementation manner provided in this embodiment, if the model to be trained is a multi-classification model, generating test data for test processing includes: Generate test data corresponding to each category through a large language model; The test data is used as input data to call the test interface of the multi-classification model for classification processing, and the test records corresponding to each data type returned by the interface call are obtained.

[0027] For example, if the multi-classification model is a takeout review classification model, the large language model can be used to generate three types of test data: "good review", "average review" and "bad review". The test data can be input into the model to be trained through the model API calling tool, and the test records returned by the model API calling tool can be obtained. The test data, evaluation indicators and test records can also be uploaded to the database through the database tool.

[0028] In this embodiment, the test record may include the original test data and the classification result of the model. Specifically, the test record may be constructed according to each input and output result. For example, the test record of the multi-classification model may be ['input data': 'classification result']. For example, if the model to be trained is a takeaway review classification model, the test record may be [('comment': 'good taste, large quantity', 'label': 'good review'), ('comment': 'the food is good, but I didn't read the notes', 'label': 'medium review'), ('comment': 'very unpalatable', 'label': 'bad review')].

[0029] In addition, after the test of the model to be trained is completed, in order to improve the user's interactive perception, a test completion message can be generated and sent to the user through the agent for the user to view. For example, the agent can generate: "The test is completed, please ask whether evaluation and optimization are required" and send it to the user for the user to view. Further, after the user interacts with the agent and inputs the evaluation and optimization instructions, steps S106 to S108 can also be executed.

[0030] Step S106, generating a test result according to the test record and the label of the test data, and determining a data type corresponding to a target indicator whose indicator in the test result is less than an indicator threshold.

[0031] In specific implementation, the test results of the model to be trained are generated according to the test records output by the model to be trained and the pre-marking of the test data, the target indicator whose indicator is less than the indicator threshold is queried in the test results, and the data type of the test data corresponding to the target indicator is determined.

[0032] In this embodiment, the test result may be composed of various model indicators calculated according to the test records and the labels of the test data, and specifically, may be composed of accuracy, recall, precision and / or F1 parameter. For example, if the model to be trained is a multi-classification model, the test result may be the accuracy, recall, precision and / or F1 parameter of the test data for each classification.

[0033] During the specific execution process, the model index and / or performance index of the model to be trained can be calculated according to the test record and the type mark of the test data, and the test result can be generated based on the model index and / or performance index, and the data type corresponding to the target index whose model index and / or performance index is less than the index threshold is determined. By calculating the model index and / or performance index, the training data of the model is determined more specifically, the model training efficiency is improved, and the model training cost is saved; in an optional implementation manner provided by this embodiment, generating the test result and determining the data type includes: Calculate the model index of each data type according to the test record and the type mark of the test data, and generate the test result based on the model index; A target indicator whose indicator is less than an indicator threshold is screened out from the model indicators, and a target data type corresponding to the target indicator is determined.

[0034] Continuing with the above example, the accuracy, duplicate detection rate, precision rate and / or F1 parameter of "good review", "average review" and "bad review" are calculated according to the test records of the takeout review classification model and the type label of the test data. The test results of the takeout review classification model are generated based on the accuracy, duplicate detection rate, precision rate and / or F1 parameter of each classification. Among the various indicators in the test results, the target indicator with an indicator less than the threshold is screened out, and the target indicator is determined to be the accuracy of "average review", then the target data type is the "average review" type.

[0035] In actual applications, there are also situations where users view the test results of the model to be trained. In view of this, the test results can be converted into evaluation results composed of natural language and sent to the user, so that the user can view the evaluation results through the terminal device, thereby improving the user's interactive experience. In an optional implementation manner provided by this embodiment, after the test results are generated, it also includes: Inputting the test result into the large language model for text generation to obtain an evaluation result of the model to be trained; The evaluation result is sent to the user through the intelligent agent; the user views the evaluation result through the terminal device.

[0036] Specifically, the test results can be data stored in the form of a data structure, and text can be generated based on the test results through a large language model, that is, the test results are converted into evaluation results composed of natural language, and the evaluation results are sent to the user. The evaluation results can be text in the form of tables and / or reports.

[0037] Step S108: Generate training data of the data type of the model to be trained by using the large language model, and train the model to be trained based on the training data.

[0038] In specific implementation, the data type is input into the large language model so that the large language model generates training data corresponding to the data type of the model to be trained, and the model to be trained is trained based on the training data to obtain a trained model. Specifically, the data type and the prompt words for generating the training data can be input into the large language model, and the training data can be input into the model to be trained through the agent tool for model training.

[0039] In the specific implementation process, if the model to be trained is a multi-classification model, training data corresponding to the data type of the target classification can be generated to improve the judgment accuracy of the model to be trained for the data of this classification. Using the above example, if the judgment accuracy of the takeaway review classification model for "medium review" does not reach the threshold, then "medium review" is input into the large language model so that the large language model generates training data with the "medium review" label, and the training data is input into the model to be trained for classification processing, and the parameters of the takeaway review classification model are adjusted according to the output results and the label "medium review".

[0040] In actual applications, there are also cases where there is no model corresponding to the keyword in the model library; in response to this, a training failure message can be generated when the model to be trained is not found, and sent to the user through the agent. For example, if the user inputs: "Please help me analyze the performance of the large model", the answer is: "Sorry, I don't have the ability to "test the large model" and I haven't found relevant information in the history records. You can ask me to test the small model."

[0041] During the specific execution process, there may be a situation where the user repeatedly performs model testing, model evaluation and / or model training. In view of this, when the historical records of the model are queried, the historical records can be returned to the user to improve the efficiency of model training. An optional implementation manner provided by this embodiment also includes: If the model test results of the model to be trained are found, the model test results are sent to the user through the agent; If the model evaluation results of the model to be trained are found, the model evaluation results will be sent to the user through the intelligent agent.

[0042] During the specific execution process, there is also a situation where the user performs model training again after the model to be trained has been trained. In view of this, when the training record of the model to be trained is queried, the training record can be sent to the user to improve data processing efficiency and enhance user perception. An optional implementation manner provided by this embodiment also includes: Obtain a secondary training request for the model to be trained, and detect whether there is a training record for the model to be trained; If not, no action will be taken; If so, the training record of the model to be trained is sent to the user through the agent.

[0043] In addition, there are cases where the user needs to test the model to be trained again after the model has been tested and optimized. In this case, a secondary test request can be generated based on the secondary test keywords contained in the model test information input by the user. For example, the user inputs: "Please test the classification model again", then the model to be trained is determined to be a classification model, and the subtask is converted to "1. Secondary test"; or, the user can also enter model training information to perform secondary optimization training on the trained model to be trained. For example, the user inputs: "Please train the classification model again", then the model to be trained is determined to be a classification model, and the subtask is converted to "1. Secondary test 2. Secondary evaluation 3. Secondary training".

[0044] It should be noted that for the model training information input by the user, the historical records can be sent to the user through the intelligent agent when the historical records exist, so that the user can view the historical records through the terminal device. When the user inputs secondary training information, the model training process can be performed again. The specific execution process can refer to the above-mentioned test, evaluation and training operations of the model to be trained, which will not be repeated here.

[0045] In summary, the one or more model training methods provided in this embodiment first obtain a model training request through the interaction between the user and the intelligent agent during the model training process, determine the model to be trained in the model library according to the keywords in the model training request, generate test data corresponding to the model type of the model to be trained through the large language model, generate targeted test data through the large language model, improve the reliability of the model test results, input the test data into the model to be trained for test processing to obtain a test record, generate a test result according to the test record and the label of the test data, determine the test result of the model by defining various indicators, generate training data for the model in a more targeted manner, improve the availability of model training, determine the data type corresponding to the target indicator whose indicator in the test result is less than the indicator threshold, finally generate training data of the data type of the model to be trained through the large language model, generate targeted training data through the large language model, train the model to be trained based on the training data, and strengthen the model targeted training for the data type, thereby improving the reliability of model training without increasing the training cost and improving the availability of the model.

[0046] The following combination Figure 2 , taking the application of the model training method provided in this embodiment in the model training scenario as an example, the model training method provided in this embodiment is further explained. Figure 2 The model training method applied to the model training scenario specifically includes steps S202 to S218.

[0047] Step S202, obtaining model training information of the model to be trained that is interactively input by the user and the agent, extracting type keywords in the model training information to generate a model training request.

[0048] Step S204: extract the type keywords of the model to be trained in the model training request, and call the query interface according to the type keywords to perform model query in the model library.

[0049] Step S206, obtaining the multi-classification model returned by the interface call, and using the multi-classification model as the model to be trained.

[0050] Step S208: Generate test data corresponding to each category through the large language model.

[0051] Step S210, calling the test interface of the multi-classification model to input test data for classification processing, and obtaining test records corresponding to each data type.

[0052] Step S212, calculating the model index of each data type according to the test record and the type mark of the test data, and generating the test result based on the model index.

[0053] Step S214, input the test results into the large language model for model evaluation, obtain the evaluation results of the model to be trained, and send the evaluation results to the user through the intelligent agent.

[0054] Step S216, select target indicators whose indicators are less than the indicator threshold value from the model indicators, and determine the data type corresponding to the target indicator.

[0055] Step S218: Generate training data of the data type of the model to be trained through the large language model, and train the model to be trained based on the training data.

[0056] It should be noted that any one of steps S202 to S218 or any combination of multiple steps can be selected from steps S102 to S108 to form a new implementation method according to the needs of implementation deployment; and any one or multiple technical features in the technical scheme composed of steps S202 to S218 can be selected from steps S102 to S108 to form a new implementation method according to the needs of actual deployment, or the technical features in one or more optional implementation methods provided by steps S102 to S108 can be selected to combine into a new implementation method, which will not be elaborated here one by one.

[0057] It should be noted that you can also refer to Figure 3 Perform optimization training processing on the model to be trained.

[0058] Figure 4A schematic diagram of a model training device provided by an embodiment of the present invention is shown in FIG. Figure 4 As shown, the device comprises: The to-be-trained model determination module 402 is used to obtain a model training request and determine a to-be-trained model in a model library according to keywords in the model training request; A model testing module 404 is used to generate test data corresponding to the model type of the model to be trained through a large language model, and input the test data into the model to be trained for testing to obtain a test record; A data type determination module 406 is used to generate a test result according to the test record and the label of the test data, and determine a data type corresponding to a target indicator whose indicator in the test result is less than an indicator threshold; The model training module 408 is used to generate training data of the data type of the model to be trained through the large language model, and train the model to be trained based on the training data.

[0059] The model training device provided in this embodiment, during the model training process, firstly runs the to-be-trained model determination module 402 to obtain the model training request through the interaction between the user and the intelligent agent, and determines the to-be-trained model in the model library according to the keywords in the model training request, and then runs the model testing module 404 to generate test data corresponding to the model type of the to-be-trained model through the large language model, and generates targeted test data through the large language model, thereby improving the reliability of the model test result, inputs the test data into the to-be-trained model for test processing to obtain a test record, runs the data type determination module 406 to generate a test result according to the test record and the label of the test data, determines the test result of the model by defining various indicators, generates the training data of the model in a more targeted manner, improves the availability of the model training, determines the data type corresponding to the target indicator whose indicator in the test result is less than the indicator threshold, and finally runs the model training module 408 to generate training data of the data type of the to-be-trained model through the large language model, generates targeted training data through the large language model, trains the to-be-trained model based on the training data, and performs targeted model training reinforcement for the data type, thereby improving the reliability of the model training without increasing the training cost, and improving the availability of the model.

[0060] The model training device provided in one embodiment of this specification can implement each process in the aforementioned method embodiment and achieve the same functions and effects, which will not be repeated here.

[0061] Furthermore, an embodiment of the present specification also provides a model training device, Figure 5 A schematic diagram of a model training device provided in an embodiment of this specification is shown in FIG. Figure 5As shown, the device includes: a memory 501, a processor 502, a bus 503 and a communication interface 504. The memory 501, the processor 502 and the communication interface 504 communicate through the bus 503, and the communication interface 504 may include an input and output interface, which includes but is not limited to a keyboard, a mouse, a display, a microphone, a loudspeaker, etc.

[0062] Figure 5 In the embodiment, the memory 501 stores computer executable instructions that can be run on the processor 502. When the computer executable instructions are executed by the processor 502, the following process is implemented: Obtain a model training request, and determine a model to be trained in a model library according to keywords in the model training request; Generate test data corresponding to the model type of the model to be trained by using a large language model, and input the test data into the model to be trained for test processing to obtain a test record; Generate a test result according to the test record and the label of the test data, and determine the data type corresponding to the target indicator whose indicator in the test result is less than the indicator threshold; The training data of the data type of the model to be trained is generated by the large language model, and the model to be trained is trained based on the training data.

[0063] The model training device provided in this embodiment cooperates with the memory 501, the processor 502, the bus 503 and the communication interface 504. During the model training process, the model training request is first obtained through the interaction between the user and the intelligent agent, the model to be trained is determined in the model library according to the keywords in the model training request, the test data corresponding to the model type of the model to be trained is generated through the large language model, and the targeted test data is generated through the large language model, thereby improving the reliability of the model test result, inputting the test data into the model to be trained for test processing to obtain a test record, generating a test result according to the test record and the label of the test data, determining the test result of the model by defining various indicators, generating the training data of the model in a more targeted manner, improving the availability of model training, determining the data type corresponding to the target indicator whose indicator in the test result is less than the indicator threshold, and finally generating the training data of the data type of the model to be trained through the large language model, generating the targeted training data through the large language model, training the model to be trained based on the training data, and strengthening the targeted model training for the data type, thereby improving the reliability of model training without increasing the training cost, and improving the availability of the model.

[0064] The model training device provided in an embodiment of this specification can implement each process in the aforementioned method embodiment and achieve the same functions and effects, which will not be repeated here.

[0065] Furthermore, another embodiment of the present specification provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and when the computer-executable instructions are executed by a processor, the following process is implemented: Obtain a model training request, and determine a model to be trained in a model library according to keywords in the model training request; Generate test data corresponding to the model type of the model to be trained by using a large language model, and input the test data into the model to be trained for test processing to obtain a test record; Generate a test result according to the test record and the label of the test data, and determine the data type corresponding to the target indicator whose indicator in the test result is less than the indicator threshold; The training data of the data type of the model to be trained is generated by the large language model, and the model to be trained is trained based on the training data.

[0066] The computer-readable storage medium provided in this embodiment first obtains a model training request through the interaction between the user and the intelligent agent during the model training process, determines the model to be trained in the model library according to the keywords in the model training request, generates test data corresponding to the model type of the model to be trained through the large language model, generates targeted test data through the large language model, improves the reliability of the model test result, inputs the test data into the model to be trained for test processing to obtain a test record, generates a test result according to the test record and the label of the test data, determines the test result of the model by defining various indicators, generates training data for the model in a more targeted manner, improves the availability of model training, determines the data type corresponding to the target indicator whose indicator in the test result is less than the indicator threshold, finally generates training data of the data type of the model to be trained through the large language model, generates targeted training data through the large language model, trains the model to be trained based on the training data, and performs targeted model training reinforcement for the data type, thereby improving the reliability of model training without increasing the training cost and improving the availability of the model.

[0067] The computer-readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0068] The computer-readable storage medium provided in an embodiment of this specification can implement each process in the aforementioned method embodiment and achieve the same functions and effects, which will not be repeated here.

[0069] Furthermore, another embodiment of the present specification also provides a computer program product, which implements the following process when executed by a processor: Obtain a model training request, and determine a model to be trained in a model library according to keywords in the model training request; Generate test data corresponding to the model type of the model to be trained by using a large language model, and input the test data into the model to be trained for test processing to obtain a test record; Generate a test result according to the test record and the label of the test data, and determine the data type corresponding to the target indicator whose indicator in the test result is less than the indicator threshold; The training data of the data type of the model to be trained is generated by the large language model, and the model to be trained is trained based on the training data.

[0070] The computer product program provided in this embodiment first obtains a model training request through the interaction between the user and the intelligent agent during the model training process, determines the model to be trained in the model library according to the keywords in the model training request, generates test data corresponding to the model type of the model to be trained through the large language model, generates targeted test data through the large language model, improves the reliability of the model test result, inputs the test data into the model to be trained for test processing to obtain a test record, generates a test result according to the test record and the label of the test data, determines the test result of the model by defining various indicators, generates training data for the model in a more targeted manner, improves the availability of model training, determines the data type corresponding to the target indicator whose indicator in the test result is less than the indicator threshold, finally generates training data of the data type of the model to be trained through the large language model, generates targeted training data through the large language model, trains the model to be trained based on the training data, and performs targeted model training reinforcement for the data type, thereby improving the reliability of model training without increasing the training cost and improving the availability of the model.

[0071] The computer program product provided in one embodiment of this specification can implement each process in the aforementioned method embodiment and achieve the same functions and effects, which will not be repeated here.

[0072] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0073] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0074] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0075] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0076] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0077] Memory may include non-permanent storage in a computer-readable storage medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable storage medium.

[0078] Computer-readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0079] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0080] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0081] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A model training method, characterized in that: The method comprises: Obtain a model training request, and determine a model to be trained in a model library according to keywords in the model training request; Generate test data corresponding to the model type of the model to be trained by using a large language model, and input the test data into the model to be trained for test processing to obtain a test record; Generate a test result according to the test record and the label of the test data, and determine the data type corresponding to the target indicator whose indicator in the test result is less than the indicator threshold; The training data of the data type of the model to be trained is generated by the large language model, and the model to be trained is trained based on the training data.

2. The model training method according to claim 1, characterized in that: Determining the model to be trained in the model library according to the keywords in the model training request includes: Extracting a type keyword of the model to be trained in the model training request, and calling a query interface according to the type keyword to perform a model query in the model library; The target model returned by the interface call is obtained, and the target model is used as the model to be trained.

3. The model training method according to claim 1, characterized in that: After generating the test result according to the test record and the label of the test data, the method further includes: Inputting the test result into the large language model for text generation to obtain an evaluation result of the model to be trained; The evaluation result is sent to the user through the intelligent agent; the user views the evaluation result through the terminal device.

4. The model training method according to claim 1, characterized in that: Generating test data corresponding to the model type of the model to be trained by using the large language model, and inputting the test data into the model to be trained for test processing to obtain a test record, includes: If the model to be trained is a multi-classification model to be trained, test data corresponding to each classification is generated by a large language model; The test data is used as input data to call the test interface of the multi-classification model for classification processing, and the test records corresponding to each data type returned by the interface call are obtained.

5. The model training method according to claim 4, characterized in that: Generating a test result according to the test record and the label of the test data, and determining a data type corresponding to a target indicator whose indicator in the test result is less than an indicator threshold, comprises: Calculate the model index of each data type according to the test record and the type label of the test data, and generate the test result based on the model index; A target indicator whose indicator is less than an indicator threshold is screened out from the model indicators, and a target data type corresponding to the target indicator is determined.

6. The model training method according to claim 1, characterized in that: After generating the training data of the data type of the model to be trained by the large language model and training the model to be trained based on the training data, the method further includes: Obtain a secondary training request for the model to be trained, and detect whether there is a training record for the model to be trained; If so, the training record of the model to be trained is sent through the agent.

7. A model training device, characterized in that: The device comprises: A module for determining a model to be trained, used to obtain a model training request and determine a model to be trained in a model library according to keywords in the model training request; A model testing module, used to generate test data corresponding to the model type of the model to be trained through a large language model, and input the test data into the model to be trained for testing and obtaining a test record; A data type determination module, configured to generate a test result according to the test record and the label of the test data, and determine a data type corresponding to a target indicator in the test result whose indicator is less than an indicator threshold; A model training module is used to generate training data of the data type of the model to be trained through the large language model, and train the model to be trained based on the training data.

8. A model training device, characterized in that: The device includes a memory and a processor, wherein the memory stores computer executable instructions, and when the computer executable instructions are executed on the processor, the steps of the model training method described in any one of claims 1 to 6 can be implemented.

9. A computer-readable storage medium having computer-executable instructions stored therein, characterized in that: When the computer executable instructions are executed by the processor, the steps of the model training method described in any one of claims 1 to 6 can be implemented.

10. A computer program product, characterized in that The computer program product includes a model training program, and when the model training program is executed by a processor, it can implement the steps of the model training method described in any one of claims 1 to 6.

Images