A model training method and device, and related equipment

Abstract

Embodiments of the present application provide a model training method and device and related equipment, and relate to the technical field of computers. The method comprises: obtaining a first reply data set, the first reply data set comprising at least two reply data; the at least two reply data being generated by a first model based on at least one question data; analyzing each reply data in the first reply data set to obtain an evaluation result corresponding to each reply data, the evaluation result being used to represent a to-be-improved condition corresponding to each reply data; determining at least one prompt word based on the evaluation result corresponding to each reply data; the prompt word being used to determine a reply data corresponding to question data; performing at least one training on the first model based on the at least one prompt word; and obtaining a second model trained under the condition that a preset condition is met.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a model training method, apparatus and related equipment. Background Technology

[0002] With the development of technology, large language models are widely used in customer service automation responses and intelligent question answering scenarios. Large language models can generate corresponding answers based on the input question.

[0003] After the large language model is deployed in real-world applications, it needs to be continuously trained to output more appropriate and accurate responses.

[0004] In related technologies, the responses output by a large language model can be evaluated manually to obtain the evaluation results of the large language model, and the large language model can be trained based on the evaluation results.

[0005] However, due to the strong subjectivity of the evaluation results obtained during the manual evaluation process, the evaluation results cannot provide accurate training and guidance for the training process of the large language model. Consequently, the response content output by the large language model trained based on the evaluation results is not accurate enough. Summary of the Invention

[0006] The purpose of this application is to provide a model training method, apparatus, and related equipment to solve the problem in related technologies where evaluation results obtained through manual evaluation are highly subjective, cannot provide accurate training and guidance for the training process of large language models, and consequently, the responses output by the large language model trained based on these evaluation results are not accurate enough. The specific technical solution is as follows:

[0007] In a first aspect of this application, a model training method is provided, the method comprising:

[0008] Obtain a first response dataset, which includes at least two response datasets; the at least two response datasets are generated by a first model based on at least one question dataset.

[0009] Analyze each response data in the first response dataset to obtain the evaluation result corresponding to each response data, and the evaluation result is used to indicate the improvement status of each response data;

[0010] Based on the evaluation results corresponding to each of the response data, at least one prompt word is determined; the prompt word is used to determine the response data corresponding to the question data.

[0011] The first model is trained at least once based on the at least one prompt word;

[0012] Under the premise of meeting the preset conditions, the trained second model is obtained.

[0013] A second aspect of this application provides a method for generating response data, comprising:

[0014] Obtain data related to the target problem;

[0015] The target response data corresponding to the target question data is determined using the second model;

[0016] The second model is a model trained based on the first aspect.

[0017] In a third aspect of this application, a model training apparatus is provided, comprising:

[0018] The acquisition module is used to acquire a first response dataset, which includes at least two response datasets; the at least two response datasets are generated by a first model based on at least one question dataset.

[0019] The evaluation module is used to analyze each response data in the first response dataset and obtain the evaluation result corresponding to each response data. The evaluation result is used to indicate the improvement status of each response data.

[0020] The first determining module is used to determine at least one prompt word based on the evaluation results corresponding to each of the response data; the prompt word is used to determine the response data corresponding to the question data.

[0021] The training module is used to train the first model at least once based on the at least one prompt word;

[0022] The second determining module is used to obtain the trained second model under preset conditions.

[0023] In a fourth aspect of this application, a response data generation apparatus is provided, comprising:

[0024] The acquisition module is used to acquire data related to the target problem.

[0025] The determination module is used to determine the target response data corresponding to the target question data through the second model;

[0026] The second model is a model trained based on the first aspect.

[0027] In a fifth aspect of this application, an electronic device is provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0028] Memory, used to store programs;

[0029] A processor, when executing a program stored in memory, implements the method described in the first aspect and the method described in the second aspect.

[0030] In a sixth aspect of this application, a readable storage medium is provided having a program stored thereon that, when executed by a processor, implements the method described in the first aspect and the method described in the second aspect.

[0031] In a seventh aspect of this application, a computer program product is provided, including computer instructions that, when executed by a processor, implement the method described in the first aspect and the method described in the second aspect.

[0032] In this embodiment, the electronic device can acquire a first response dataset and evaluate each response data in the dataset to obtain an evaluation result for each response data, thereby determining the current state of improvement of the first model's response data. Based on the evaluation results for each response data, the electronic device then determines at least one prompt word to train the first model at least once, resulting in a trained first model, which is the second model. During this process, the electronic device can automatically generate objective evaluation results and, based on these results, generate prompt words that provide accurate training and guidance. This allows the second model, after being trained on the first model using the prompt words, to output more suitable and accurate response data. Furthermore, this application eliminates the need for manual determination of evaluation results, improving processing efficiency and reducing labor costs. Attached Figure Description

[0033] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0034] Figure 1 This is one of the flowcharts illustrating a model training method in an embodiment of this application;

[0035] Figure 2 This is a second schematic flowchart of a model training method in an embodiment of this application;

[0036] Figure 3 This is a flowchart illustrating a response data generation method according to an embodiment of this application;

[0037] Figure 4 This is a schematic diagram of the structure of a model training device according to an embodiment of this application;

[0038] Figure 5 This is a schematic diagram of the structure of a response data generation device according to an embodiment of this application;

[0039] Figure 6 This is a structural diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0040] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0041] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. A process can be terminated when its operation is complete, but it may also have additional steps not included in the figures. A process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0042] Furthermore, the terms "first," "second," etc., may be used herein to describe various directions, actions, steps, or elements, but these directions, actions, steps, or elements are not limited by these terms. These terms are only used to distinguish a first direction, action, step, or element from another direction, action, step, or element. For example, without departing from the scope of this application, a first speed difference may be referred to as a second speed difference, and similarly, a second speed difference may be referred to as a first speed difference. Both the first speed difference and the second speed difference are speed differences, but they are not the same speed difference. The terms "first," "second," etc., should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0043] This application provides a model training method, such as... Figure 1 As shown, the steps of this method include:

[0044] Step 101: Obtain a first response dataset, which includes at least two response datasets; the at least two response datasets are generated by a first model based on at least one question dataset.

[0045] In some embodiments, the electronic device may acquire at least two response data generated by a first model based on at least one question data, wherein each question data may correspond to at least two response data generated by the first model.

[0046] For example, assuming there are n questions, for each question in the n questions, the electronic device can obtain at least two response data generated by the first model for each question. That is, the first response dataset can include at least 2n response data, and at least 2n response data correspond to n questions.

[0047] Step 102: Analyze each response data in the first response dataset to obtain the evaluation result corresponding to each response data. The evaluation result is used to indicate the improvement status corresponding to each response data.

[0048] In some embodiments, the evaluation result includes: at least one evaluation analysis result and a total evaluation score, wherein analyzing each response data in the first response dataset to obtain the evaluation result corresponding to each response data includes:

[0049] For each response data in the first response dataset, the response data is analyzed based on the target evaluation dimension to obtain the evaluation analysis result corresponding to each evaluation dimension in the target evaluation dimension. The evaluation analysis result is used to indicate the improvement status of the response data in the corresponding evaluation dimension.

[0050] Determine the evaluation score corresponding to each evaluation analysis result, wherein the evaluation score is used to represent the completeness score corresponding to the evaluation analysis result;

[0051] The total evaluation score of the response data is obtained by calculating the evaluation score corresponding to each evaluation analysis result of a response data.

[0052] In this embodiment, the target evaluation dimension can include one or more evaluation dimensions. For example, the target evaluation dimension may include: accuracy dimension, language fluency dimension, and recommendation effect dimension. The accuracy dimension measures the degree to which the response data matches the facts, the language fluency dimension measures the naturalness and fluency of the response data, and the recommendation effect dimension measures the degree to which the recommended content matches and personalizes the user's needs. In this process, the electronic device can extract and structure multiple core evaluation dimensions based on the experience and feedback of human evaluation, forming a refined scoring standard that can be understood and executed by a large language model. In some embodiments, the evaluation dimensions used may differ in different question-and-answer scenarios. The electronic device can adjust and optimize the evaluation according to different scenarios, adding evaluation dimensions or changing the quantification method to adapt to more diverse needs.

[0053] For each response in the first response dataset, the electronic device determines the evaluation analysis result for each evaluation dimension based on preset evaluation criteria. For example, the preset evaluation criteria might be: accuracy dimension: whether there are errors in the basic information of the response data; fluency dimension: whether the sentences in the response data are fluent; and recommendation effectiveness dimension: whether the recommended content in the response data is appropriate. Then, based on the preset evaluation criteria, the electronic device might obtain the following evaluation analysis result for response data 1 in the first response dataset: accuracy dimension: the director information mentioned in response data 1 is incorrect; fluency dimension: the sentences in response data 1 are fluent; and recommendation effectiveness dimension: response data 1 recommends similar films but lacks personalization.

[0054] A score range can be set for each evaluation dimension to quantify and score each dimension. The total evaluation score for the response data is then calculated based on the quantified scores for each dimension. The calculation method can be additive, weighted, etc., and no limitation is made in the embodiments of this application.

[0055] For example, the evaluation scores for accuracy, fluency, and recommendation effectiveness can each range from 0 to 2. Based on the evaluation analysis of response data 1 above, the electronic device can assign a score of 0 for accuracy, 2 for fluency, and 1 for recommendation effectiveness. The electronic device then adds up the scores for each dimension to obtain a total evaluation score of 3.

[0056] In some embodiments, the electronic device can also use a large language model to generate a quantitative score for each evaluation dimension, and another large language model to generate the evaluation analysis result corresponding to each evaluation dimension. This avoids the bias caused by using only one large language model to obtain both the evaluation analysis result and the total evaluation score, thus improving the objectivity and fairness of the evaluation results. That is, the electronic device can integrate and fuse the evaluation results of multiple different large language models, thereby improving the robustness of the evaluation.

[0057] After determining the evaluation analysis results and scores for each evaluation dimension in the target evaluation dimension, as well as the total evaluation score of the response data, the electronic device can use a post-processing program (e.g., regular expressions) to parse the evaluation analysis results, scores, and total evaluation scores of the response data, and store them separately.

[0058] Understandably, in this embodiment, the electronic device can analyze the response data from the target evaluation dimensions to determine the areas for improvement, and determine the corresponding evaluation score based on the evaluation analysis results of the target evaluation dimensions, thereby determining the total evaluation score of the response data. This process facilitates subsequent optimization and training of the first model generating the response data based on the evaluation analysis results and scores for each evaluation dimension in the target evaluation dimensions, and the total evaluation score of the response data. Furthermore, this process not only provides the total evaluation score but also accurately identifies the deficiencies in the response data across specific evaluation dimensions, facilitating fine-grained identification of subsequent issues requiring improvement and comprehensively assessing the quality of the first model's output, providing quantitative indicators for subsequent evaluation and optimization.

[0059] Step 103: Based on the evaluation results corresponding to each response data, determine at least one prompt word; the prompt word is used to determine the response data corresponding to the question data.

[0060] In some embodiments, determining at least one prompt word based on the evaluation results corresponding to each of the response data includes:

[0061] Obtain the target response data from the first response dataset to obtain the second response dataset. The target response data is the response data whose total evaluation score is less than or equal to a preset evaluation threshold.

[0062] Based on each evaluation analysis result corresponding to each target response data in the second response dataset, the at least one prompt word is determined.

[0063] In this embodiment, the electronic device can determine the response data whose total evaluation score is less than or equal to a preset evaluation threshold, and obtain a second response dataset. The electronic device then acquires the evaluation analysis results corresponding to each response data in the second response dataset, and determines at least one prompt word based on the evaluation analysis results corresponding to each response data in the second response dataset.

[0064] Understandably, in this embodiment, the electronic device can determine at least one prompt word based on each evaluation analysis result corresponding to each target response data in the second response dataset, so as to facilitate subsequent training and optimization of the first model using the prompt word.

[0065] In some embodiments, determining the at least one prompt word based on each evaluation analysis result corresponding to each target response data in the second response dataset includes:

[0066] Based on each evaluation analysis result corresponding to each target response data in the second response dataset, the problems to be improved corresponding to the first model are determined, and the problems to be improved are used to indicate the problems existing in the first model.

[0067] The problem to be improved is input into the prompt word optimization model to obtain at least one prompt word output by the prompt word optimization model. The prompt word is used to train the first model to solve the existing problem.

[0068] In this embodiment, the electronic device can utilize text clustering algorithms or keyword extraction algorithms to determine the issues to be improved in the first model based on each evaluation analysis result corresponding to each target response data in the second response dataset. For example, the issues to be improved could be: factual errors in the response data, or homogeneity in the recommended content in the response data. The text clustering algorithm can be a combination of term frequency–inverse document frequency (TF-IDF) and K-means clustering.

[0069] For example, the electronic device can determine the problem to be improved using a text clustering algorithm as follows: The electronic device first obtains the evaluation analysis result corresponding to each target response data in the second response dataset. Then, in response to a score in the evaluation analysis result being lower than a preset threshold, the electronic device determines the response data corresponding to the score being lower than the preset threshold as the text set to be analyzed.

[0070] The electronic device then employs a term frequency-inverse document frequency algorithm to extract features from each text in the text set to be analyzed, converting each text into a corresponding text feature vector. This text feature vector is used to characterize the keyword distribution features of the corresponding text. The electronic device then uses a K-means clustering algorithm to cluster all text feature vectors, dividing the text set to be analyzed into at least one text cluster based on the spatial distance between the vectors. The texts within each cluster exhibit high similarity at the keyword level.

[0071] The electronic device then performs semantic analysis on each text cluster, statistically analyzing the high-frequency keywords and their co-occurrence relationships within that cluster, and determines the issues to be addressed based on the statistical results. For example, if a text cluster contains high-frequency keywords such as "fact," "error," and "inaccuracy," the issue to be addressed for that cluster can be identified as "factual errors exist in the response data." If another text cluster contains high-frequency keywords such as "recommendation," "homogenization," and "similarity," the issue to be addressed for that cluster can be identified as "homogenization of recommended content in the response data." Thus, by mapping the clustering results to the preset problem categories, an automatic transformation from quantitative scoring to qualitative issues is achieved.

[0072] For example, the electronic device can determine the issues to be improved using a keyword extraction algorithm as follows: First, the electronic device obtains the evaluation analysis results corresponding to each target response data in the second response dataset. Then, in response to a score in the evaluation analysis results that is lower than a preset threshold, the electronic device identifies the response data corresponding to the score below the preset threshold as the text set to be analyzed. The electronic device then uses a keyword extraction algorithm to extract keywords from all texts in the text set to be analyzed, obtaining at least one keyword corresponding to each text, and counting the frequency of each keyword in the text set to be analyzed. The electronic device sorts the keywords in descending order according to their frequency of occurrence, and identifies the top N keywords with the highest frequency as high-frequency question keywords.

[0073] The electronic device then matches high-frequency question keywords with a pre-set question mapping library, which stores the correspondence between keywords and questions to be improved. For example, if the frequency of "fact" and "error" is significantly higher than expected among high-frequency question keywords, the corresponding question to be improved is determined to be "factual errors exist in the response data"; if the frequency of "recommendation" and "homogenization" is significantly higher than expected, the corresponding question to be improved is determined to be "homogenization of recommended content in the response data". Through this method, the electronic device extracts keyword features from low-scoring response data and automatically determines the question to be improved corresponding to the first model based on the statistical regularity of keyword frequency, realizing a quantifiable and reproducible technical path from scoring to question category.

[0074] The electronic device then uses a prompt word optimization model to determine the prompt words corresponding to the problem to be improved. This prompt word optimization model can be a large language model. The electronic device can input the problem to be improved, the current prompt words, and relevant cases for prompt word optimization into the prompt word optimization model, obtaining modified candidate prompt words output by the model to solve the problem. For example, the prompt word optimization model can be obtained through algorithms such as reinforcement learning to improve the convergence speed and effectiveness of prompt word generation and optimization.

[0075] For example, an electronic device can input the "issue that needs improvement due to frequent factual errors in the current prompt words and response data" into the prompt word optimization model, and obtain a new prompt word output by the prompt word optimization model: "Please prioritize retrieving and verifying information from the knowledge base."

[0076] Understandably, in this embodiment, the electronic device can determine at least one prompt word based on the problem to be improved, which facilitates the subsequent training and optimization of the first model using at least one prompt word. This eliminates the need for manual determination of at least one prompt word based on the problem to be improved, reducing manual operations and achieving automated improvement and optimization of the prompt word.

[0077] In some embodiments, the method further includes:

[0078] Based on the issues that need improvement, we have identified the evaluation dimensions to be added.

[0079] The target evaluation dimension is updated based on the evaluation dimensions to be added.

[0080] In this embodiment, after identifying the problem to be improved, the electronic device can also add new evaluation dimensions based on the problem to be improved, update the current target evaluation dimensions, and determine the evaluation analysis results of the response data on the newly added evaluation dimensions in the subsequent process of determining the evaluation analysis results corresponding to each evaluation dimension in the target evaluation dimensions of the response data.

[0081] For example, suppose the current target evaluation dimensions include accuracy, fluency, and recommendation effectiveness. The issue to be addressed is the homogeneity of recommended content in the response data. The electronic device can then add a new evaluation dimension, such as diversified recommendation. This diversified recommendation dimension is then added to the target evaluation dimensions. During the subsequent process of determining the evaluation analysis results for each evaluation dimension in the target evaluation dimensions of the response data, the electronic device will obtain the evaluation analysis results for accuracy, fluency, recommendation effectiveness, and the newly added diversified recommendation dimension.

[0082] Understandably, in this embodiment, the electronic device determines the new evaluation dimension based on the problem to be improved, and the target evaluation dimension is based on the new evaluation dimension. This makes it easier for the electronic device to determine the evaluation analysis result corresponding to the response data on the new evaluation dimension, so as to determine whether the problem to be improved has been solved. That is, it is easier to determine whether the response data output by the trained first model still has problems to be improved.

[0083] Step 104: Train the first model at least once based on the at least one prompt word.

[0084] The prompt words can be the content of strategy optimization for the first model. For example, the prompt words can be: "Please retrieve and verify the information from the knowledge base first", which is used to instruct the first model to retrieve and verify the response data from the knowledge base before generating the response data, so as to avoid factual errors in the generated response data.

[0085] In some embodiments, training the first model at least once based on the at least one prompt word includes:

[0086] The first model is trained N times based on at least one prompt word;

[0087] The nth training session is any one of the N training sessions, and during the nth training session, prompt words are added to the first model after the (n-1)th training session to obtain the first model after the nth training session.

[0088] In this embodiment, the electronic device can train the first model using prompts, and the generated prompts can be one or more. Before each training iteration of the first model, the electronic device can add one or more prompts to the first model to obtain the trained model.

[0089] For example, assuming there is only one prompt word, the electronic device can add that prompt word to the first model and perform multiple training iterations to obtain the trained first model. Alternatively, if there are multiple prompt words, the electronic device can add one prompt word during the first training iteration of the first model and perform the first training iteration to obtain the first trained first model. Then, it can add another new prompt word during the second training iteration of the first model and perform the second training iteration to obtain the second trained first model, and so on. Or, if there are multiple prompt words, the electronic device can add multiple prompt words to the first model simultaneously and perform multiple training iterations to obtain the trained first model.

[0090] Understandably, in this embodiment, the electronic device can train and optimize the first model by adding prompt words to the first model, so that the trained first model outputs more appropriate and accurate response data.

[0091] Step 105: Under the premise of meeting the preset conditions, the trained second model is obtained.

[0092] In some embodiments, the method further includes:

[0093] Determine the third total evaluation score corresponding to the first model after the N-1th training, and the fourth total evaluation score corresponding to the first model after the Nth training. The third total evaluation score is the average of the total evaluation scores corresponding to the response data output by the first model after the N-1th training, and the fourth total evaluation score is the average of the total evaluation scores corresponding to the response data output by the first model after the Nth training.

[0094] Determine the difference between the total score of the third evaluation and the total score of the fourth evaluation;

[0095] The preset conditions include at least one of the following: the difference is less than or equal to a preset convergence threshold; or the total score of the third evaluation is greater than the total score of the fourth evaluation.

[0096] The process of obtaining the trained second model under preset conditions includes:

[0097] If the difference is less than or equal to the preset convergence threshold, or if the total score of the third evaluation is greater than the total score of the fourth evaluation, the first model after the (N-1)th training iteration is used as the second model.

[0098] In this embodiment, the electronic device can determine the average value of the total evaluation score of the first model after each training session. That is, after each training session, the electronic device re-inputs the question data into the trained first model to obtain new response data, determines the total evaluation score corresponding to each response data, and performs an average calculation to determine the average value of the total evaluation score of the trained first model.

[0099] The electronic device then determines the average total score of the first model after the previous training and the average total score of the first model after the current training, and calculates the difference between the two averages.

[0100] If the difference is greater than the preset convergence threshold, it proves that the prompt words used in the current training of the first model are effective and can optimize and improve the accuracy of the response data output by the first model. The electronic device adopts the prompt words used in the current training and uses them as a benchmark to continue the training of the first model in the next training.

[0101] If the difference is less than or equal to the preset convergence threshold, or if the average total score of the first model after the previous training is greater than or equal to the average total score of the first model after the current training, it indicates that the training of the first model has been completed and the training of the first model has converged, and the first model after the previous training can be used as the second model.

[0102] If the stopping condition for training the first model is triggered, the currently trained first model can also be used as the second model. The stopping condition can be: the number of training iterations is greater than or equal to a preset training threshold, or the average total evaluation score of the first model after this training iteration is greater than a preset threshold, etc., and no restrictions are imposed here.

[0103] Understandably, in this embodiment, the electronic device determines whether the first model meets preset conditions by comparing the average total evaluation score of the first model after the previous training iteration with the average total evaluation score of the first model after the current training iteration, and then uses these averages to determine the second model that meets the training requirements. This process automates the determination of the first model's training convergence, allowing the electronic device to selectively output a second model that meets the training requirements.

[0104] As can be seen from steps 101-105 above, in this process, the electronic device can automatically generate objective evaluation results and, based on these results, generate prompts that can provide accurate training and guidance. This allows the electronic device to train the first model using these prompts, resulting in a second model that can output more suitable and accurate response data, providing users with a better natural language interaction experience. Furthermore, the embodiments of this application can automatically and efficiently complete evaluation and optimization, accelerating the optimization and iteration speed of the first model, significantly reducing manpower input, shortening the deployment cycle, and lowering continuous iteration costs.

[0105] In some embodiments, the method further includes:

[0106] Determine the first total score for each of the at least two response data points corresponding to the first question data;

[0107] Based on the preset scoring sort, determine the second total evaluation score corresponding to at least two response data for each response data in the first question data;

[0108] Based on the first total evaluation score, determine the first rating ranking result of at least two response data corresponding to the first question data;

[0109] Based on the second total evaluation score, determine the second rating ranking result of at least two response data corresponding to the first question data;

[0110] The first rating ranking result and the second rating ranking result are compared to determine whether the first rating ranking result and the second rating ranking result are consistent.

[0111] Optionally, during the stage of acquiring the first response dataset, the electronic device can also acquire and determine a preset scoring order among at least two response data corresponding to each question data. This preset scoring order is determined based on the scores given by human evaluators to at least two response data corresponding to each question data in the first response dataset. For example, assuming there is question data 1 (or the first question data), the electronic device uses a first model to determine response data 1 and response data 2 corresponding to question data 1, and scores response data 1 and response data 2 to obtain the first total evaluation score for each. Then, the first scoring order is determined based on the first total evaluation scores for each of response data 1 and response data 2. The electronic device then uses the preset scoring order to determine the second total evaluation score for each of response data 1 and response data 2 corresponding to question data 1, and then determines the second scoring order based on the second total evaluation scores for each of response data 1 and response data 2. The electronic device can also compare the preset scoring order to determine whether the first scoring order and the second scoring order are consistent. If they are inconsistent, the first scoring order and the second scoring order are aligned. Electronic devices can also verify the alignment effect through the Kendall-Tau coefficient to determine whether the sorting result corresponding to the preset scoring sort is the same as or consistent with the sorting result corresponding to the scoring sort of the electronic device.

[0112] In this embodiment, in addition to training the first model through the evaluation results, the electronic device can also train the first model through the preset score ranking and evaluation results between at least two response data corresponding to each question data. The preset score ranking is a relative ranking, that is, it is a relative score ranking between at least two response data corresponding to each question data. The relative ranking is more stable and reliable than the evaluation score of a single response data corresponding to each question data, and can effectively eliminate personal scoring bias.

[0113] Optionally, determining at least one prompt word based on the evaluation results corresponding to each of the response data includes:

[0114] Based on the preset scoring order, the total evaluation score corresponding to each response data is determined;

[0115] Obtain at least one evaluation analysis result corresponding to each of the response data;

[0116] The total evaluation score corresponding to each response data, at least one evaluation analysis result corresponding to each response data, and each response data are input into the prompt word optimization model to obtain the at least one prompt word output by the prompt word optimization model.

[0117] For example, after determining the preset rating ranking, the electronic device can use a preset prompt word optimization model to embed samples (such as evaluation results, evaluation criteria, response data in the first response dataset, and the preset rating ranking corresponding to the response data) into the prompt word optimization model to obtain prompt words. The electronic device then uses the prompt words to perform zero-sample or small-sample learning and training to guide the first model to understand and apply the evaluation results and evaluation criteria, thus obtaining the second model.

[0118] For example, electronic devices can fine-tune the first model using a contrastive learning loss function (such as Pairwise Ranking Loss) to obtain a second model.

[0119] In some embodiments, the electronic device can input question data into a second model to obtain at least two response data corresponding to the question data output by the second model. The electronic device then determines whether the sorting of the at least two response data output by the second model is the same as a preset scoring sorting to determine whether the first model has been trained successfully, that is, whether the response data output by the second model is appropriate and accurate. If the sorting is the same, the first model has been trained successfully, and the response data output by the second model is appropriate and accurate. If the sorting is different, the first model has not been trained successfully, and the response data output by the second model is inappropriate and inaccurate.

[0120] The electronic device can determine the sorting method corresponding to at least two response data points output by the second model in the following ways: The electronic device converts each response data point output by the second model into a vector representation corresponding to each response data point, determines the score corresponding to each response data point through the vector representation, and determines the sorting method based on the score corresponding to each response data point. Alternatively, the electronic device can determine the score corresponding to each response data point, i.e., determine the total evaluation score corresponding to each response data point, based on the aforementioned method of determining the evaluation analysis results and evaluation scores corresponding to each evaluation dimension in the target evaluation dimension.

[0121] In this embodiment, by inputting the total evaluation score corresponding to each response data, at least one evaluation analysis result corresponding to each response data determined based on a preset scoring order, and each response data into the prompt word optimization model, the at least one prompt word output by the prompt word optimization model can be automatically obtained. This overcomes the bottleneck of low efficiency in manual evaluation and optimization in related technologies and is conducive to quickly and efficiently optimizing the performance of the prompt word optimization model.

[0122] Next, we will proceed through... Figure 2 The training process of the first model in this application is described in detail.

[0123] Step 201: Obtain the first response dataset and evaluate each response data in the first response dataset to obtain the evaluation result corresponding to each response data.

[0124] Step 202: Determine the response data in the evaluation results whose total evaluation score is less than or equal to the preset evaluation threshold, and generate a second response dataset.

[0125] Step 203: Based on each evaluation analysis result corresponding to each target response data in the second response dataset, determine the problem to be improved corresponding to the first model, and determine at least one prompt word based on the problem to be improved.

[0126] Step 204: Train the first model using at least one prompt word, and determine whether the trained first model meets the preset conditions.

[0127] Step 205: If the first model after training does not meet the preset conditions, return to step 201 to regenerate prompt words based on the evaluation results corresponding to the response data.

[0128] Step 206: If the first trained model meets the preset conditions, output the first trained model, i.e., the second model.

[0129] As can be seen from the above steps 201-206, in this application, the electronic device will transform the identified specific problems to be improved into targeted and optimizable prompt words, and use the prompt words to train the first model, so that the second model trained by the first model can output more accurate and appropriate response data.

[0130] In some embodiments, the electronic device can also receive new response data output by the trained first model and re-input it into the target evaluation dimension for scoring. By comparing the score changes before and after the first model training, a closed-loop feedback loop of "evaluation-localization-optimization-validation" is formed. In this process, the effectiveness of each prompt word modification can be objectively and quantitatively judged, thereby supporting rapid iteration and significantly improving the optimization efficiency and effectiveness of prompt words.

[0131] like Figure 3 The present application also provides a response data generation method, the steps of which include:

[0132] Step 301: Obtain target problem data.

[0133] Step 302: Determine the target response data corresponding to the target question data using the second model.

[0134] The second model is based on the above. Figure 1-2 The model obtained through training.

[0135] In this embodiment, the second model can output more accurate and appropriate response data.

[0136] like Figure 4 As shown in the illustration, this application also provides a model training device 400, comprising:

[0137] The acquisition module 401 is used to acquire a first response dataset, the first response dataset including: at least two response data; the at least two response data are generated by a first model based on at least one question data;

[0138] Evaluation module 402 is used to analyze each response data in the first response dataset to obtain the evaluation result corresponding to each response data, and the evaluation result is used to indicate the improvement status corresponding to each response data.

[0139] The first determining module 403 is used to determine at least one prompt word based on the evaluation results corresponding to each of the response data; the prompt word is used to determine the response data corresponding to the question data.

[0140] Training module 404 is used to train the first model at least once based on the at least one prompt word;

[0141] The second determining module 405 is used to obtain the trained second model under the condition that the preset conditions are met.

[0142] Optionally, the evaluation results include: at least one evaluation analysis result and a total evaluation score; the evaluation module 402 may further include:

[0143] An evaluation unit is used to analyze each response data in the first response dataset based on a target evaluation dimension, and to obtain an evaluation analysis result corresponding to each evaluation dimension in the target evaluation dimension. The evaluation analysis result is used to indicate the improvement status of the response data in a corresponding evaluation dimension.

[0144] The first determining unit is used to determine the evaluation score corresponding to each of the evaluation analysis results, wherein the evaluation score is used to represent the completeness score corresponding to the evaluation analysis result.

[0145] The first calculation unit is used to calculate the evaluation score corresponding to each evaluation analysis result of a response data to obtain the total evaluation score of the response data.

[0146] Optionally, the first determining module 403 may further include:

[0147] The first acquisition unit is used to acquire target response data in the first response dataset and obtain a second response dataset, wherein the target response data is response data whose total evaluation score is less than or equal to a preset evaluation threshold;

[0148] The second determining unit is used to determine the at least one prompt word based on each evaluation analysis result corresponding to each target response data in the second response dataset.

[0149] Optionally, the second determining unit may further include:

[0150] The first determining subunit is used to determine the problem to be improved corresponding to the first model based on each evaluation analysis result corresponding to each target response data in the second response dataset, wherein the problem to be improved is used to indicate the problem existing in the first model.

[0151] The second determining subunit is used to input the problem to be improved into the prompt word optimization model to obtain at least one prompt word output by the prompt word optimization model, and the prompt word is used to train the first model to solve the existing problem.

[0152] Optionally, the first determining module 403 may further include:

[0153] The third determining unit is used to determine the total evaluation score corresponding to each response data based on a preset scoring sort.

[0154] The second acquisition unit is used to acquire at least one evaluation analysis result corresponding to each of the response data;

[0155] The input unit is used to input the total evaluation score corresponding to each response data, at least one evaluation analysis result corresponding to each response data, and each response data into the prompt word optimization model to obtain the at least one prompt word output by the prompt word optimization model.

[0156] Optionally, the model training device 400 may also include:

[0157] The third determining module is used to determine the evaluation dimensions to be added based on the issues to be improved.

[0158] The update module is used to update the target evaluation dimension based on the evaluation dimension to be added.

[0159] Optionally, training module 404 may also include:

[0160] The first training unit is used to train the first model N times based on the at least one prompt word;

[0161] The nth training session is any one of the N training sessions, and during the nth training session, the parameters of the first model after the (n-1)th training session are adjusted to obtain the first model after the nth training session. The parameters are determined based on the at least one prompt word.

[0162] Optionally, the model training device 400 may also include:

[0163] The fourth determining module is used to determine the third total evaluation score corresponding to the first model after the N-1th training and the fourth total evaluation score corresponding to the first model after the Nth training. The third total evaluation score is the average of the total evaluation scores corresponding to the response data output by the first model after the N-1th training, and the fourth total evaluation score is the average of the total evaluation scores corresponding to the response data output by the first model after the Nth training.

[0164] The fifth determining module is used to determine the difference between the total score of the third evaluation and the total score of the fourth evaluation;

[0165] The preset conditions include at least one of the following: the difference is less than or equal to a preset convergence threshold; or the total score of the third evaluation is greater than the total score of the fourth evaluation.

[0166] The second determining module 405 may also include:

[0167] The fourth determining unit is used to take the first model after the N-1th training as the second model when the difference is less than or equal to the preset convergence threshold, or when the total score of the third evaluation is greater than the total score of the fourth evaluation.

[0168] The model training device 400 provided in this application embodiment can achieve... Figure 1 The various processes implemented in the method embodiments shown are capable of achieving the same beneficial effects, and will not be described again here to avoid repetition.

[0169] like Figure 5 As shown in the illustration, this application also provides a response data generation apparatus 500, comprising:

[0170] Module 501 is used to acquire target problem data;

[0171] The determination module 502 is used to determine the target response data corresponding to the target question data through the second model;

[0172] The second model is based on the above. Figure 1-2 The model obtained through training.

[0173] The response data generation device 500 provided in this application embodiment can achieve... Figure 3The various processes implemented in the method embodiments shown are capable of achieving the same beneficial effects, and will not be described again here to avoid repetition.

[0174] This application also provides an electronic device, such as... Figure 6 As shown, it includes a processor 601, a communication interface 602, a memory 603, and a communication bus 604, wherein the processor 601, the communication interface 602, and the memory 603 communicate with each other through the communication bus 604.

[0175] The memory 603 is used to store computer programs; the processor 601, when executing the program stored in the memory 603, performs the following steps:

[0176] Obtain a first response dataset, which includes at least two response datasets; the at least two response datasets are generated by a first model based on at least one question dataset.

[0177] Analyze each response data in the first response dataset to obtain the evaluation result corresponding to each response data, and the evaluation result is used to indicate the improvement status of each response data;

[0178] Based on the evaluation results corresponding to each of the response data, at least one prompt word is determined; the prompt word is used to determine the response data corresponding to the question data.

[0179] The first model is trained at least once based on the at least one prompt word;

[0180] Under the premise of meeting the preset conditions, the trained second model is obtained.

[0181] Optionally, the evaluation result includes: at least one evaluation analysis result and a total evaluation score. The analysis of each response data point in the first response dataset to obtain the evaluation result corresponding to each response data point includes:

[0182] For each response data in the first response dataset, the response data is analyzed based on the target evaluation dimension to obtain the evaluation analysis result corresponding to each evaluation dimension in the target evaluation dimension. The evaluation analysis result is used to indicate the improvement status of the response data in the corresponding evaluation dimension.

[0183] Determine the evaluation score corresponding to each evaluation analysis result, wherein the evaluation score is used to represent the completeness score corresponding to the evaluation analysis result;

[0184] The total evaluation score of the response data is obtained by calculating the evaluation score corresponding to each evaluation analysis result of a response data.

[0185] Optionally, the processor 601 is further configured to, when executing the program stored in the memory 603, implement the following steps: determining at least one prompt word based on the evaluation results corresponding to each of the response data, including:

[0186] Obtain the target response data from the first response dataset to obtain the second response dataset. The target response data is the response data whose total evaluation score is less than or equal to a preset evaluation threshold.

[0187] Based on each evaluation analysis result corresponding to each target response data in the second response dataset, the at least one prompt word is determined.

[0188] Optionally, the processor 601 is further configured to, when executing the program stored in the memory 603, implement the following steps: determining the at least one prompt word based on each evaluation analysis result corresponding to each target response data in the second response dataset, including:

[0189] Based on each evaluation analysis result corresponding to each target response data in the second response dataset, the problems to be improved corresponding to the first model are determined, and the problems to be improved are used to indicate the problems existing in the first model.

[0190] The problem to be improved is input into the prompt word optimization model to obtain at least one prompt word output by the prompt word optimization model. The prompt word is used to train the first model to solve the existing problem.

[0191] Optionally, the processor 601 is further configured to, when executing the program stored in the memory 603, implement the following steps: determining at least one prompt word based on the evaluation results corresponding to each of the response data, including:

[0192] Based on the preset scoring order, the total evaluation score corresponding to each response data is determined;

[0193] Obtain at least one evaluation analysis result corresponding to each of the response data;

[0194] The total evaluation score corresponding to each response data, at least one evaluation analysis result corresponding to each response data, and each response data are input into the prompt word optimization model to obtain the at least one prompt word output by the prompt word optimization model.

[0195] Optionally, when executing a program stored in memory 603, processor 601 may also perform the following steps:

[0196] Based on the issues that need improvement, we have identified the evaluation dimensions to be added.

[0197] The target evaluation dimension is updated based on the evaluation dimensions to be added.

[0198] Optionally, the processor 601 is further configured to, when executing the program stored in the memory 603, implement the following steps: training the first model at least once based on the at least one prompt word, including:

[0199] The first model is trained N times based on at least one prompt word;

[0200] The nth training session is any one of the N training sessions, and during the nth training session, the parameters of the first model after the (n-1)th training session are adjusted to obtain the first model after the nth training session. The parameters are determined based on the at least one prompt word.

[0201] Optionally, when executing a program stored in memory 603, processor 601 may also perform the following steps:

[0202] Determine the third total evaluation score corresponding to the first model after the N-1th training, and the fourth total evaluation score corresponding to the first model after the Nth training. The third total evaluation score is the average of the total evaluation scores corresponding to the response data output by the first model after the N-1th training, and the fourth total evaluation score is the average of the total evaluation scores corresponding to the response data output by the first model after the Nth training.

[0203] Determine the difference between the total score of the third evaluation and the total score of the fourth evaluation;

[0204] The preset conditions include at least one of the following: the difference is less than or equal to a preset convergence threshold; or the total score of the third evaluation is greater than the total score of the fourth evaluation.

[0205] The process of obtaining the trained second model under preset conditions includes:

[0206] If the difference is less than or equal to the preset convergence threshold, or if the total score of the third evaluation is greater than the total score of the fourth evaluation, the first model after the (N-1)th training iteration is used as the second model.

[0207] The memory 603 is used to store computer programs; the processor 601, when executing the program stored in the memory 603, can also perform the following steps:

[0208] Obtain data related to the target problem;

[0209] The target response data corresponding to the target question data is determined using the second model.

[0210] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not indicate that there is only one bus or one type of bus.

[0211] The communication interface is used for communication between the aforementioned terminal and other devices.

[0212] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0213] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0214] In another embodiment provided in this application, a computer-readable storage medium is also provided, which stores instructions that, when executed on a computer, cause the computer to perform any of the model training methods and response data generation methods described in the above embodiments.

[0215] In another embodiment provided in this application, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the model training methods and response data generation methods described in the above embodiments.

[0216] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0217] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0218] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0219] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application are included within the scope of protection of this application.

Claims

1. A model training method, characterized in that, The method includes: Obtain a first response dataset, which includes at least two response datasets; the at least two response datasets are generated by a first model based on at least one question dataset. Analyze each response data in the first response dataset to obtain the evaluation result corresponding to each response data, and the evaluation result is used to indicate the improvement status of each response data; Based on the evaluation results corresponding to each of the response data, at least one prompt word is determined; the prompt word is used to determine the response data corresponding to the question data. The first model is trained at least once based on the at least one prompt word; Under the premise of meeting the preset conditions, the trained second model is obtained.

2. The method of claim 1, wherein, The evaluation results include: at least one evaluation analysis result and a total evaluation score. The analysis of each response data point in the first response dataset to obtain the evaluation result corresponding to each response data point includes: For each response data in the first response dataset, the response data is analyzed based on the target evaluation dimension to obtain the evaluation analysis result corresponding to each evaluation dimension in the target evaluation dimension. The evaluation analysis result is used to indicate the improvement status of the response data in the corresponding evaluation dimension. Determine the evaluation score corresponding to each evaluation analysis result, wherein the evaluation score is used to represent the completeness score corresponding to the evaluation analysis result; The total evaluation score of the response data is obtained by calculating the evaluation score corresponding to each evaluation analysis result of a response data.

3. The method of claim 2, wherein, The step of determining at least one prompt word based on the evaluation results corresponding to each of the response data includes: Obtain the target response data from the first response dataset to obtain the second response dataset. The target response data is the response data whose total evaluation score is less than or equal to a preset evaluation threshold. Based on each evaluation analysis result corresponding to each target response data in the second response dataset, the at least one prompt word is determined.

4. The method of claim 3, wherein, The step of determining the at least one prompt word based on each evaluation analysis result corresponding to each target response data in the second response dataset includes: Based on each evaluation analysis result corresponding to each target response data in the second response dataset, the problems to be improved corresponding to the first model are determined, and the problems to be improved are used to indicate the problems existing in the first model. The problem to be improved is input into the prompt word optimization model to obtain at least one prompt word output by the prompt word optimization model. The prompt word is used to train the first model to solve the existing problem.

5. The method of claim 2, wherein, The step of determining at least one prompt word based on the evaluation results corresponding to each of the response data includes: Based on the preset scoring order, the total evaluation score corresponding to each response data is determined; Obtain at least one evaluation analysis result corresponding to each of the response data; The total evaluation score corresponding to each response data, at least one evaluation analysis result corresponding to each response data, and each response data are input into the prompt word optimization model to obtain the at least one prompt word output by the prompt word optimization model.

6. The method according to claim 5, characterized in that, The method further includes: Determine the first total score for each of the at least two response data points corresponding to the first question data; Based on the preset scoring sort, determine the second total evaluation score corresponding to at least two response data for each response data in the first question data; Based on the first total evaluation score, determine the first rating ranking result of at least two response data corresponding to the first question data; Based on the second total evaluation score, determine the second rating ranking result of at least two response data corresponding to the first question data; The first rating ranking result and the second rating ranking result are compared to determine whether the first rating ranking result and the second rating ranking result are consistent.

7. The method according to claim 4, characterized in that, The method further includes: Based on the issues that need improvement, we have identified the evaluation dimensions to be added. The target evaluation dimension is updated based on the evaluation dimensions to be added.

8. The method according to claim 2, characterized in that, The step of training the first model at least once based on the at least one prompt word includes: The first model is trained N times based on at least one prompt word; The nth training session is any one of the N training sessions, and during the nth training session, the parameters of the first model after the (n-1)th training session are adjusted to obtain the first model after the nth training session. The parameters are determined based on the at least one prompt word.

9. The method according to claim 7, characterized in that, The method further includes: Determine the third total evaluation score corresponding to the first model after the N-1th training, and the fourth total evaluation score corresponding to the first model after the Nth training. The third total evaluation score is the average of the total evaluation scores corresponding to the response data output by the first model after the N-1th training, and the fourth total evaluation score is the average of the total evaluation scores corresponding to the response data output by the first model after the Nth training. Determine the difference between the total score of the third evaluation and the total score of the fourth evaluation; The preset conditions include at least one of the following: the difference is less than or equal to a preset convergence threshold; or, the total score of the third evaluation is greater than the total score of the fourth evaluation; obtaining the trained second model under the preset conditions includes: If the difference is less than or equal to the preset convergence threshold, or if the total score of the third evaluation is greater than the total score of the fourth evaluation, the first model after the (N-1)th training iteration is used as the second model.

10. A method for generating response data, characterized in that, include: Obtain data related to the target problem; The target response data corresponding to the target question data is determined using the second model; The second model is a model trained by the method according to any one of claims 1-9.

11. A model training device, characterized in that, include: The acquisition module is used to acquire a first response dataset, which includes at least two response datasets; the at least two response datasets are generated by a first model based on at least one question dataset. The evaluation module is used to analyze each response data in the first response dataset and obtain the evaluation result corresponding to each response data. The evaluation result is used to indicate the improvement status of each response data. The first determining module is used to determine at least one prompt word based on the evaluation results corresponding to each of the response data; the prompt word is used to determine the response data corresponding to the question data. The training module is used to train the first model at least once based on the at least one prompt word; The second determining module is used to obtain the trained second model under preset conditions.

12. A response data generation apparatus, characterized in that, include: The acquisition module is used to acquire data related to the target problem. The determination module is used to determine the target response data corresponding to the target question data through the second model; The second model is a model trained by the method according to any one of claims 1-9.

13. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store programs; A processor, when executing a program stored in memory, implements the method as described in any one of claims 1-9, and the method as described in claim 10.

14. A readable storage medium having a program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-9, and the method as described in claim 10.

15. A computer program product, characterized in that, Includes computer instructions that, when executed by a processor, implement the method as described in any one of claims 1-9, and the method as described in claim 10.

Images