Model training method, data processing method, electronic device, computer readable storage medium and computer program product

Abstract

The invention discloses a model training method, a data processing method, electronic equipment, a computer readable storage medium and a computer program product, and relates to the technical field of large model technology and data processing. The model training method comprises the following steps: training an initial model by adopting first training data to obtain a first training result; evaluating the first training result from a plurality of evaluation dimensions to obtain feedback information; second training data is constructed based on the feedback information and multi-role information, and the multi-role information is used for setting a plurality of different roles in the training data construction process; and performing iterative adjustment on the initial model by adopting the second training data to obtain a target model. The technical problems that the model training efficiency is low and the effect is poor due to the fact that the degree of dependence of training data set construction on manpower is high are solved.

Description

Technical Field

[0001] The present application relates to the fields of large model technology and data processing technology, and specifically, to a model training method, a data processing method, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] In the field of natural language processing, large models, with their powerful learning and reasoning capabilities, are increasingly becoming core tools for performing a variety of tasks, such as conversation summarization, work order classification, and other tasks. However, with the increasing complexity and sophistication of business needs, how to efficiently and accurately improve the fine-tuning effect of large models has become a technical problem that needs to be solved urgently. The traditional fine-tuning process relies on manually constructed model fine-tuning datasets. The process of building this dataset not only consumes a lot of time and manpower, but also often requires technicians to manually evaluate and manually optimize each iteration of the model. In other words, the existing model fine-tuning training methods that rely on manual intervention are difficult to meet the needs of rapid model iteration and performance improvement in application scenarios.

[0003] Therefore, how to improve the model training mechanism to enhance the efficiency and effect of model training has become one of the important technical issues in the relevant technical field. No effective solution has been proposed so far for the above problems. Summary of the invention

[0004] The embodiments of the present application provide a model training method, a data processing method, an electronic device, a computer-readable storage medium, and a computer program product to at least solve the technical problem of low model training efficiency and poor effect caused by the high degree of manual dependence on the construction of training data sets.

[0005] According to one aspect of an embodiment of the present application, a model training method is provided, including: using first training data to train an initial model to obtain a first training result; evaluating the first training result from multiple evaluation dimensions to obtain feedback information; constructing second training data based on the feedback information and multi-role information, wherein the multi-role information is used to set multiple different roles in the training data construction process; and iteratively adjusting the initial model using the second training data to obtain a target model.

[0006] According to another aspect of an embodiment of the present application, a data processing method is also provided, including: obtaining a target problem to be answered; analyzing the target problem using a target model to generate a target answer; wherein the target model is obtained by iteratively adjusting the initial model using target training data, the target training data is constructed based on feedback information and multi-role information, the multi-role information is used to set multiple different roles in the training data construction process, the feedback information is obtained by evaluating the initial training results from multiple evaluation dimensions, and the initial training results are obtained by training the initial model using the initial training data.

[0007] According to another aspect of an embodiment of the present application, a data processing method is also provided, including: obtaining customer service questions to be answered; analyzing the customer service questions using a target customer service model to generate customer service answers; wherein the target customer service model is obtained by iteratively adjusting the initial customer service model using target customer service training data, the target customer service training data is constructed based on customer service feedback information and multi-role information, the multi-role information is used to set multiple different customer service roles in the process of constructing the customer service training data, the customer service feedback information is obtained by evaluating the initial customer service training results from multiple evaluation dimensions, and the initial customer service training results are obtained by training the initial customer service model using the initial customer service training data.

[0008] According to another aspect of an embodiment of the present application, a data processing method is also provided, including: obtaining a data processing request through a first application programming interface, wherein the request data carried in the data processing request includes: a target question to be answered; returning a data processing response through a second application programming interface, wherein the response data carried in the data processing response includes: a target answer, wherein the target answer is obtained by analyzing the target question using a target model, wherein the target model is obtained by iteratively adjusting the initial model using target training data, wherein the target training data is constructed based on feedback information and multi-role information, wherein the multi-role information is used to set multiple different roles in the process of constructing training data, wherein the feedback information is obtained by evaluating the initial training results from multiple evaluation dimensions, and wherein the initial training results are obtained by training the initial model using the initial training data.

[0009] According to another aspect of an embodiment of the present application, a data processing method is also provided, including: obtaining a currently input data processing dialogue request, wherein the request data carried in the data processing dialogue request includes: a target question to be answered; in response to the data processing dialogue request, returning a data processing dialogue reply, wherein the information carried in the data processing dialogue reply includes: a target answer, the target answer is obtained by analyzing the target question using a target model, the target model is obtained by iteratively adjusting the initial model using target training data, the target training data is constructed based on feedback information and multi-role information, the multi-role information is used to set multiple different roles in the training data construction process, the feedback information is obtained by evaluating the initial training results from multiple evaluation dimensions, and the initial training results are obtained by training the initial model using the initial training data; and displaying the target answer in a graphical user interface.

[0010] According to another aspect of an embodiment of the present application, an electronic device is also provided, including: a memory storing an executable program; and a processor for running the program, wherein any one of the above-mentioned model training methods is executed when the program is running.

[0011] According to another aspect of an embodiment of the present application, a computer-readable storage medium is also provided, and the computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned model training methods.

[0012] According to another aspect of an embodiment of the present application, a computer program product is also provided, including a computer program, which implements any of the above-mentioned model training methods when executed by a processor.

[0013] In an embodiment of the present application, the initial model is trained using the first training data to obtain a first training result; the first training result is evaluated from multiple evaluation dimensions to obtain feedback information; the second training data is further constructed based on the feedback information and the multi-role information, and the multi-role information is used to set multiple different roles in the training data construction process; and the initial model is iteratively adjusted using the second training data to obtain a target model. Based on this, the present application evaluates the pre-trained model, and comprehensively considers the multiple different roles set in the training data construction process, and automatically constructs the second training data for model fine-tuning iteration. Since the feedback information obtained from the above evaluation can characterize the training effect of the first training result of the initial model from multiple evaluation dimensions, and the automatically constructed second training data is guided by the above feedback information and multi-role information, the embodiment of the present application can reduce the dependence of the training data construction process on manual labor, the construction basis of the training data is more comprehensive, and the training mechanism of the target model is optimized, so as to achieve the purpose of automatically constructing the model fine-tuning training data and reducing the dependence of the model training process on manual labor, thereby achieving the technical effect of improving the automation degree and comprehensiveness of the training data construction process, and improving the training efficiency and training effect of the target model, thereby solving the technical problem of low model training efficiency and poor effect caused by the high degree of manual dependence on the construction of the training data set.

[0014] It is easy to notice that the above general description and the following detailed description are only for the purpose of exemplifying and explaining the present application, and do not constitute a limitation of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0016] Figure 1 is a schematic diagram of an application scenario of a model training method according to an embodiment of the present application;

[0017] Figure 2 is a flow chart of a model training method according to an embodiment of the present application;

[0018] Figure 3 is a schematic diagram of an optional model training process according to an embodiment of the present application;

[0019] Figure 4 is a flow chart of a data processing method according to an embodiment of the present application;

[0020] Figure 5 is a flow chart of another data processing method according to an embodiment of the present application;

[0021] Figure 6 is a flow chart of another data processing method according to an embodiment of the present application;

[0022] Figure 7 is a flow chart of another data processing method according to an embodiment of the present application;

[0023] Figure 8 is a structural schematic diagram of a model training device according to an embodiment of the present application;

[0024] Fig. 9 is a structural schematic diagram of a data processing device according to an embodiment of the present application;

[0025] Fig.10 It is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0027] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0028] The technical solution provided in this application is mainly implemented using large-scale model technology. The large model here refers to a deep learning model with large-scale model parameters, which can usually contain hundreds of millions, tens of billions, hundreds of billions, trillions or even more than 10 trillion model parameters. The large model can also be called a foundation model / foundation model. The large model is pre-trained with large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks, and the model has good generalization ability, such as large-scale language model (Large Language Model, LLM for short), multi-modal pre-training model, etc.

[0029] It should be noted that when the large model is actually applied, the pre-trained model can be fine-tuned through a small number of samples so that the large model can be applied to different tasks. For example, the large model can be widely used in natural language processing (NLP), computer vision, speech processing and other fields, and can be specifically applied to computer vision tasks such as visual question answering (VQA), image caption (IC), image generation, etc. It can also be widely used in natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. Therefore, the main application scenarios of the large model include but are not limited to digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc. In the embodiment of the present application, data processing through an intelligent dialogue large language model in a customer service scenario is used as an example for explanation.

[0030] First, some nouns or terms that appear in the process of describing the embodiments of the present application are subject to the following explanations.

[0031] Model fine-tuning (Scalable Fine-Tuning, SFT for short): also known as "scalable fine-tuning", refers to the process of retraining a pre-trained neural network model for a specific task on limited supervised data (that is, labeled data sets). Model fine-tuning can make the model more adaptable to the specific needs of a specific task and improve the performance of the model on the specific task.

[0032] Preference Ranking Optimization (PRO): refers to the process of fine-tuning the model by learning the human preference ranking of different model output results to achieve supervised adjustment of model training and ensure that the model output is aligned with human preferences. Preference ranking optimization can optimize the quality of model output in a supervised manner.

[0033] Bad cases: refers to cases that the model fails to process correctly or give reasonable answers to during the training or application of the machine learning model. These bad cases can reflect the defects of the model. The model can be optimized and corrected based on the bad cases to achieve model repair for the defects, thereby improving the accuracy and reliability of the model.

[0034] In-Context Learning (ICL) refers to a prompt engineering strategy that uses task instances in natural language as part of the prompt. By understanding examples and instructions in the context, the model can better understand the task requirements and improve its performance on new or similar tasks without the need for a lot of additional training directly on the new task.

[0035] Data synthesis: refers to the process of using algorithms to automatically generate additional training data for model training. These training data can cover scenarios that may be missed in model training or enhance learning in specific aspects, thereby improving the generalization ability of the model and the accuracy of handling complex tasks. Through data synthesis, the learning effect of the model can be effectively enhanced, especially when data is scarce or specific types of data are required.

[0036] According to an embodiment of the present application, a model training method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] Considering the huge amount of model parameters of the large model and the limited computing resources of the mobile terminal, the above model training method provided in the embodiment of the present application can be applied to Figure 1 The application scenarios shown are not limited to these. Figure 1In the application scenario shown, the large model is deployed in the server 10, and the server 10 can be connected to one or more client devices 20 through a local area network connection, a wide area network connection, an Internet connection, or other types of data networks. The client device 20 here may include but is not limited to: a smart phone, a tablet computer, a laptop computer, a PDA, a personal computer, a smart home device, a vehicle-mounted device, etc. The client device 20 can interact with the user through a graphical user interface to implement the call of the large model, thereby implementing the method provided in the embodiment of the present application.

[0038] Under the above operating environment, this application provides Figure 2 The model training method shown. Figure 2 is a flow chart of a model training method according to an embodiment of the present application, such as Figure 2 As shown, the model training method includes the following steps S21 to S24.

[0039] Step S21, using first training data to train the initial model to obtain a first training result.

[0040] The first training data may be a data set used for initial training of the initial model. The first training data may include multiple input-output pairs, which can help the model learn and establish the association between input and output. For example, in a customer service scenario, the first training data may include past customer consultation records (such as questions input by customers) and corresponding model processing results (such as answers output by intelligent customer service).

[0041] The initial model may be a model that has not been trained or has only been preliminarily trained, and the initial model contains a series of unadjusted parameters and structures. For example, in a customer service scenario, the initial model may be a conversation model that is built based on a general language model (such as a large language model) and has not yet been optimized for a specific customer service scenario.

[0042] The first training result is the training result obtained after the initial model is trained using the first training data. The first training result may include an intermediate model obtained after adjusting the model parameters of the initial model. The first training result may also include performance indicators of the intermediate model (such as accuracy, recall, etc.) and prediction results of the intermediate model on the test set. In the customer service scenario, the first training result can be used to characterize the initial performance of the dialogue model after preliminary training when handling a series of customer questions (for example, the dialogue model's understanding of the question and the quality of the answer given).

[0043] In an application scenario, according to the above steps, the above first training data is a pre-constructed evaluation set, which includes multiple groups of samples, each group of samples includes a data pair of "customer question, customer service response". The customer question is input into the initial model, and the predicted response output by the initial model is obtained. Then, the training loss is determined based on the predicted response and the customer response, and then the model parameters of the initial model are adjusted based on the training loss to obtain the first training result. The first training result at least includes the predicted responses corresponding to the multiple groups of samples.

[0044] Step S22, evaluating the first training result from multiple evaluation dimensions to obtain feedback information.

[0045] The above-mentioned multiple evaluation dimensions can be standards for evaluating model performance from different perspectives. For example, multiple evaluation dimensions can include correctness dimension, completeness dimension, format dimension, and instruction compliance dimension. For example, in a customer service scenario, multiple evaluation dimensions can include the accuracy of the model's predicted answers, whether the answer content is complete, whether the answer format meets the requirements (such as whether it contains greetings, whether polite language is used, etc.), and whether the model can correctly follow specific business processes or instructions to give answers.

[0046] The feedback information may be instructive information formed based on the evaluation of the first training result, and the feedback information is used to indicate in which aspects the model performs well and in which aspects it needs to be improved. For example, in a customer service scenario, the feedback information may include the common types of errors made by the model when handling customer questions, the types of defects in customer service answers, and conversation scenarios that may require more training data.

[0047] In the application scenario, according to the above steps, the first training result is evaluated to determine the performance of the model after preliminary training on a specific task. The above evaluation process may involve comparing the prediction result in the first training result with a preset standard or a true answer, calculating the performance indicator score according to multiple evaluation dimensions, and thus determining feedback information.

[0048] It is easy to notice that, when multiple evaluation dimensions are pre-set, the system can automatically evaluate the first training results of the initial model based on the multiple evaluation dimensions, so as to comprehensively and objectively characterize the initial training effect of the initial model, accurately and quickly determine the aspects of the model to be improved through feedback information, and help provide guidance for the subsequent training data construction.

[0049] Step S23, constructing second training data based on the feedback information and the multi-role information, wherein the multi-role information is used to set a plurality of different roles in the process of constructing the training data.

[0050] The above multi-role information can be information about different actors and / or different participants that are considered when constructing training data. For example, in a customer service scenario, the multi-role information can be used to set different types of customer roles (such as technical customers, sales customers, etc.) and different types of customer service roles (such as technical support customer service, sales customer service, etc.).

[0051] The second training data is a data set used to fine-tune the initial model. The second training data is constructed based on feedback information and multi-role information, taking into account the aspects of the model to be improved represented by the feedback information, and setting multiple different roles through multi-role information to cover a wider range of communication scenarios and communication styles in application scenarios. In a customer service scenario, the second training data may contain specific types of customer questions and dialogue responses from different customer service roles. These customer questions can be selected based on feedback information, which can at least characterize the aspects of improvement (i.e., performance weaknesses) shown by the initial model in the initial training using the first training data.

[0052] In the application scenario, according to the above steps, the second training data is designed and generated by analyzing the feedback information and the multi-role information. The construction process of the second training data can be implemented by using data synthesis technology, data enhancement technology or data set recombination technology based on feedback information and multi-role information to ensure that the model can learn from a wider range of perspectives and more complex scenarios during the model fine-tuning stage based on the second training data, thereby improving the model performance.

[0053] Step S24, iteratively adjusting the initial model using the second training data to obtain a target model.

[0054] The above-mentioned iterative adjustment refers to the process of iteratively fine-tuning the model parameters of the initial model for multiple rounds using the second training data. Specifically, the initial model is trained for multiple rounds using multiple training samples included in the second training data, and the model parameters are adjusted according to the model output after each round of training, so as to gradually improve the performance of the model in processing specific tasks. The iterative adjustment process may include multiple iterations until the model reaches a predetermined performance indicator or until further training cannot significantly improve the model performance.

[0055] The above target model can be a model that has been iteratively trained to reach the expected performance level. In the customer service scenario, during the iterative adjustment of the initial model, the model will continuously learn and optimize its ability to handle customer issues. The trained target model can efficiently and accurately understand customer issues and provide appropriate answers, while being able to adapt to different types of customers and service scenarios, thereby improving customer satisfaction and processing efficiency.

[0056] Through the above method steps, the embodiment of the present application provides a closed-loop model training process, and by continuously training, evaluating, adjusting and optimizing the initial model, a target model with excellent performance in multiple evaluation dimensions is finally obtained. The second training data of the target model is automatically generated by considering feedback information and multi-role information, which reduces the dependence of the training data construction process on manual labor, improves the efficiency of the model training process, and reduces the training cost.

[0057] In an embodiment of the present application, for example, a system composed of a client device and a server can perform the following steps: the client device executes training of an initial model using first training data to obtain a first training result; the server executes evaluation of the first training result from multiple evaluation dimensions to obtain feedback information, and constructs second training data based on the feedback information and multi-role information, wherein the multi-role information is used to set multiple different roles in the training data construction process; further, the client device executes iterative adjustment of the initial model using the second training data to obtain a target model. For another example, the client device provides the first training data and the initial model to the server, and the server executes training of the initial model using the first training data to obtain a first training result, evaluates the first training result from multiple evaluation dimensions to obtain feedback information, constructs second training data based on the feedback information and multi-role information, and iteratively adjusts the initial model using the second training data to obtain a target model; further, the server provides the trained target model to the client device or supports the client device to use it through an interface.

[0058] It should be noted that, when the operating resources of the client device can meet the deployment and operating conditions of the large model, the embodiments of the present application can be carried out in the client device.

[0059] In an embodiment of the present application, the pre-trained model is evaluated, and the multiple different roles set in the training data construction process are fully considered to automatically construct the second training data for model fine-tuning iteration. Since the feedback information obtained from the above evaluation can characterize the training effect of the first training result of the initial model from multiple evaluation dimensions, and the automatically constructed second training data is guided by the above feedback information and multi-role information, the embodiment of the present application can reduce the dependence of the training data construction process on manual labor, the construction basis of the training data is more comprehensive, and the training mechanism of the target model is optimized, so as to achieve the purpose of automatically constructing the model fine-tuning training data and reducing the dependence of the model training process on manual labor, thereby achieving the technical effect of improving the automation degree and comprehensiveness of the training data construction process and improving the training efficiency and training effect of the target model, thereby solving the technical problem of low model training efficiency and poor effect caused by the high degree of dependence on manual labor in the construction of the training data set.

[0060] The model training method provided in the embodiment of the present application has the original advantages of multi-dimensional evaluation and feedback-based iterative adjustment, and can be widely used in multiple technical fields, especially in scenarios that require processing large amounts of unstructured data, performing deep learning or natural language processing.

[0061] For example, in the field of natural language processing, the model training method provided in the embodiment of the present application is particularly suitable for intelligent customer service systems. By analyzing user conversation history and model training feedback information, the responsiveness of the target model is continuously optimized, thereby significantly improving service quality and user satisfaction. In text generation and writing assistance scenarios, the target model can adjust the style, accuracy and grammar of the generated text based on feedback to ensure that the output content is more in line with specific contexts or needs. In terms of sentiment analysis and opinion mining, the model training method provided in the embodiment of the present application can enhance the target model's ability to recognize and understand human emotions or opinions, making the target model more accurate in analyzing text sentiment and mining user opinions.

[0062] For example, in the field of machine learning and deep learning, the above method can be applied to image recognition and processing tasks, and continuously improve the recognition and processing performance of the target model through multi-dimensional feedback (such as accuracy, recall rate, etc.). In the field of speech recognition and synthesis, the above method can optimize the speech recognition accuracy and naturalness of the target model, while improving the quality and realism of speech synthesis, providing users with a more natural and smooth voice interaction experience. In terms of recommendation systems, the above method can dynamically adjust the recommendation algorithm corresponding to the target model based on user interaction and feedback data, providing users with more personalized and accurate recommendation content to meet the diverse needs of users.

[0063] For example, in the field of artificial intelligence education, the above method can help intelligent teaching assistants (i.e., target models) use students' learning feedback and guidance from different teacher roles to adjust teaching strategies and content, provide students with more effective learning assistance, and promote personalized learning.

[0064] For example, in the field of medical health, the above method can be applied to medical diagnosis assistance. By analyzing the doctor's diagnosis results and patient feedback, the diagnostic accuracy of the target model can be continuously optimized. It is also applicable to health consultation and disease prediction, providing patients with more reliable services.

[0065] For example, in the financial field, the above method can help intelligent investment advisors (also known as target models) continuously adjust investment advice strategies based on market feedback and user investment behavior, improve the accuracy of investment decisions, and help investors grasp market dynamics.

[0066] For example, in the field of Internet of Things and smart home, the above method can help smart assistants (i.e., target models) adjust the control strategies of smart home devices through user feedback, improve user experience and the intelligent response capabilities of devices, and make smart homes more intimate and efficient. In the field of autonomous driving, the above method can help the path planning and decision-making algorithms of the target model to continuously optimize and adjust according to road conditions and driving safety feedback, improve the safety and efficiency of autonomous driving, and provide protection for future travel.

[0067] The model training method provided in the embodiments of the present application can be applied to, but is not limited to, the technical fields listed above. With the continuous evolution of technology, the above method can also be applied to a wider range of fields, such as legal consultation, artistic creation, scientific research, etc., so that the model can adapt to the ever-changing environment and needs through continuous learning and improvement, and provide users with more intelligent and accurate services.

[0068] The following is a further explanation of other optional implementation steps included in the above method steps provided in the embodiment of the present application, taking the customer service application scenario as an example.

[0069] In an optional embodiment, in step S22, the first training result is evaluated from multiple evaluation dimensions to obtain feedback information, including the following method steps:

[0070] Step S221, evaluating the first training result from multiple evaluation dimensions to obtain evaluation results, wherein the evaluation results include: correct results and incorrect results;

[0071] Step S222, obtaining result generation experience based on the correct results, and determining the model evolution direction based on the wrong results;

[0072] Step S223, determining feedback information based on result generation experience and model evolution direction.

[0073] The first training result may be a result obtained after the initial model is preliminarily trained using the first training data, that is, the first training result includes the prediction result given by the initial model for the sample problem in the first training data. The evaluation result may be a result obtained by evaluating the prediction result under multiple evaluation dimensions. The evaluation result includes a correct result and a mismatch result.

[0074] The above correct results include multiple correct samples (i.e. Good cases). If the difference between the model's prediction result for a certain training sample and the corresponding sample answer meets the preset quality requirements in multiple evaluation dimensions, then the training sample and the corresponding prediction result are regarded as a correct sample.

[0075] The above-mentioned error results include multiple error samples (i.e., Badcases). If the difference between the model’s prediction result for a certain training sample and the corresponding sample answer does not meet the preset quality requirements in multiple evaluation dimensions, the training sample and the corresponding prediction result will be considered as an error sample.

[0076] In the above optional embodiment, the initial training results (i.e., the first training results) of the initial model are evaluated in multiple dimensions, and it is recorded which predicted results of the model during the initial training process match the actual results (i.e., the sample answers) or the expected results to obtain the correct results; and which predicted results do not match the actual results or the expected results to obtain the wrong results. For example, in a customer service scenario, the system may evaluate whether the initial model's answers to user questions are accurate, whether they comply with polite language standards, whether the solutions provided are effective, and whether they are timely, thereby obtaining an evaluation result that includes correct answers and wrong answers.

[0077] The above-mentioned result generation experience can be a pattern, rule or strategy extracted from the correct result. The result generation experience is used to guide the model to better make correct prediction results when dealing with problems similar to the correct samples in the correct result.

[0078] The above-mentioned model evolution direction can be the aspect of the model that needs to be improved or enhanced based on the analysis of the error results. The model evolution direction is used to guide the model to reduce wrong predictions and improve overall performance when dealing with problems similar to the error results.

[0079] In the above optional embodiments, the correct results are used to learn and summarize which operations, information or strategies help the model produce correct prediction results, and obtain result generation experience. At the same time, the wrong results are analyzed to find out the shortcomings or misunderstandings of the model, so as to determine the direction of model evolution. For example, in a customer service scenario, if the model's answer to a question is always accurate and polite, then the system can summarize the pattern that the model should follow when dealing with similar questions; if the model's answers to certain questions are often biased or key information is omitted, the system will determine that the model needs to improve its understanding of these questions and information extraction.

[0080] The above feedback information determines the information of the model fine-tuning training process to be fed back to the initial model based on the result generation experience and model evolution direction corresponding to the initial training results of the initial model. The above feedback information is used to guide the construction of training data in the model fine-tuning training process, and then guide the iterative training in the model fine-tuning process. For example, in a customer service application scenario, according to the preliminary training results, it is found that the model performs well in polite language, but has deficiencies in providing solutions. The feedback information may include more training samples on the effectiveness of the solution, as well as possible adjustment parameters or training strategies to encourage the model to improve in providing solutions while maintaining the model's good performance in polite language.

[0081] Through the above-mentioned steps S221 to S223, the embodiment of the present application comprehensively evaluates the preliminary training results of the initial model, realizes efficient identification and classification of model training problems, and realizes targeted feedback information construction, which provides accurate and effective guidance for the iterative adjustment of the model, helps to accelerate the optimization process of the model, and improves the final performance of the model.

[0082] In an optional embodiment, in step S221, the first training result is evaluated from multiple evaluation dimensions to obtain an evaluation result, including the following method steps:

[0083] Step S2211, determining multiple evaluation indicators from multiple evaluation dimensions, wherein the multiple evaluation indicators include at least one of the following: an evaluation indicator of the correctness of the evaluation result, an evaluation indicator of the integrity of the evaluation result, an evaluation indicator of the format of the evaluation result, and an evaluation indicator of the instruction compliance of the evaluation result;

[0084] Step S2212: Evaluate the first training result based on at least some of the multiple evaluation indicators to obtain an evaluation result.

[0085] The above multiple evaluation indicators are specific quantitative evaluation standards corresponding to multiple evaluation dimensions, and each evaluation indicator is used to measure the performance of the model on the corresponding evaluation dimension. Correctness evaluation indicators are used to evaluate the accuracy of the model prediction results (i.e., the model output answer). For example, for the customer service question "What is the battery capacity of this mobile phone?", the output answer of the customer service model should be the accurate battery capacity value. Completeness evaluation indicators are used to evaluate the comprehensiveness of the information provided in the model output answer. For example, for the customer service question "What colors are available for this mobile phone?", the output answer of the customer service model should not only list all the colors of the mobile phone, but also list all currently available colors to ensure the comprehensiveness of the information. Format evaluation indicators are used to evaluate whether the format of the model output answer meets the preset standards. For example, if the scenario requires the model output to be in lightweight data exchange (JavaScript Object Notation, referred to as JSON) format, then the format of the model output answer should comply with the JSON format. Instruction compliance evaluation indicators are used to evaluate whether the model correctly understands and executes the user's instructions in the process of predicting the output answer. For example, for the customer service question "Please list all mobile phone models in order from low to high by price", the customer service model needs to correctly sort the mobile phone models.

[0086] In the above optional embodiment, the system will determine multiple specific evaluation indicators based on multiple preset evaluation dimensions. In the process of determining multiple evaluation indicators, it may involve designing and selecting appropriate indicators to comprehensively evaluate the performance of the model, ensuring that the model not only performs well in a single dimension, but also meets the expected standards in multiple dimensions.

[0087] In the above optional embodiment, the system will conduct a detailed evaluation of the first training result based on multiple evaluation indicators. The evaluation process may include automated indicator calculations, such as accuracy, recall, format matching, etc.; the evaluation process may also include manual evaluation. By combining the scores of different evaluation indicators, the system can fully understand the current performance level and deficiencies of the model.

[0088] Through the above steps S2211 to S2212, the embodiment of the present application conducts a comprehensive and detailed evaluation of the training effect of the initial model, focusing not only on the correctness of the model's answers, but also on the comprehensiveness of the answers, the standardization of the output format, and the correct understanding and execution of user instructions. Based on this, the embodiment of the present application can ensure that the model meets high standards in multiple aspects (corresponding to multiple evaluation dimensions), and improve the effectiveness of the model in actual application scenarios and user experience. For example, in a customer service scenario, the embodiment of the present application helps to ensure that the customer service model can accurately understand and respond to user needs, provide complete, correctly formatted answers that meet user expectations, thereby improving the efficiency and satisfaction of customer service.

[0089] In an optional embodiment, in step S222, determining the model evolution direction based on the error result includes the following method steps:

[0090] Step S2221, clustering the error results to obtain a first clustering result;

[0091] Step S2222, determining a target subset according to the first clustering result;

[0092] Step S2223, starting from the cluster center of the target subset, a group traversal method is used to determine the model evolution direction corresponding to the target subset.

[0093] The above-mentioned error results may include multiple model output results (i.e., multiple error samples) that fail to meet expectations or standards as determined by evaluating the initial training results of the initial model. For example, in a customer service scenario, an error result may be an inaccurate, incomplete, or unformatted output answer given by the customer service model when answering a customer's question.

[0094] The first clustering result is a clustering result obtained by clustering multiple error samples in the error result. Specifically, an unsupervised learning method is used to cluster and group multiple error samples according to a preset clustering algorithm or a preset similarity metric, so that error samples in the same group are similar to each other, while error samples in different groups are more different.

[0095] The above-mentioned preset clustering algorithms may include but are not limited to: density-based spatial clustering (Density-Based Spatial Clustering of Applications with Noise, referred to as DBSCAN) algorithm, K-means clustering algorithm, hierarchical clustering algorithm, graph-based clustering algorithm, spectral clustering algorithm and density peak clustering algorithm, etc.

[0096] In the above optional embodiments, the error results are analyzed using a clustering algorithm based on the features such as the content, type or cause of the error samples, to find out the common patterns or categories of the error samples, and then the error results are divided into multiple groups. For example, the customer service model may frequently make mistakes when handling a specific type of problem (such as refund process consultation). At this time, the error samples related to the specific type of problem can be grouped into one group because these error samples may be related to the customer service model's insufficient understanding of the refund policy.

[0097] The target subset may be one or more specific clustering groups selected from the first clustering result. The target subset is used to determine the subsequent model evolution direction. For example, the target subset may be selected based on the severity, frequency of occurrence, or impact on the business of the error samples in multiple groups.

[0098] In the above optional embodiment, based on the clustering results, the clustering groups with higher error rates, greater impact on customer experience, or higher priority that need to be improved are identified as target subsets. By selecting the target subset, it is possible to ensure that training data is constructed in a more efficient way and model performance is improved.

[0099] For example, in the error sample group related to the refund process, if it is found that the customer service model often omits key document information required for refunds (which has a higher priority for improvement because missing key information may directly lead to the customer's inability to smoothly complete the refund process), then this error sample group will be identified as the target subset.

[0100] The cluster center is a representative error sample in each cluster group of the target subset, which is usually an error sample with the smallest average distance (here, the distance in the feature space) to other error samples in the cluster group.

[0101] After determining the target subset, starting from the cluster center of each cluster group in the target subset, analyze all the error samples in each cluster group one by one (that is, the group traversal method), understand the source, nature and image of the error sample, and then determine the model evolution direction corresponding to the target subset. The model evolution direction is used to determine in which aspects the model needs to be improved, such as increasing data training for specific types of problems, adjusting algorithm parameters or optimizing model structure.

[0102] For example, in the customer service scenario, in the target subset related to the refund process, starting from the cluster center of the cluster group "missing key document information", traverse all the wrong answers in the cluster group, and analyze the reasons why the customer service model misses key document information. For example, the reason may be that the customer service model does not understand specific keywords enough, or the customer service model does not fully understand the steps of the refund process. Through detailed analysis, it can be determined that the customer service model needs to strengthen training in document information recognition and process understanding, thereby clarifying the model evolution direction of the customer service model.

[0103] Through the above-mentioned steps S2221 to S2223, the embodiment of the present application can systematically and specifically identify and understand the problems existing in the initial model after preliminary training (i.e., the various error types obtained by clustering error results), especially the error types that have a greater impact on the application scenarios and have a higher priority to be solved, and then determine the direction of model evolution, which helps to build more accurate and effective training data, and guide the initial model to iteratively adjust in a specific direction, thereby significantly improving the accuracy and efficiency of the model when dealing with similar problems.

[0104] In an optional embodiment, in step S2223, starting from the cluster center of the target subset, a group traversal method is used to determine the model evolution direction corresponding to the target subset, including the following method steps:

[0105] Step S2224, starting from the cluster center of the target subset, the target subset is traversed in groups multiple times according to the similarity between the error results contained in the target subset and the preset group size, and during the group traversal process, the traversal analysis result of the previous group is set as the traversal analysis input of the next group, until all groups are traversed and the model evolution direction corresponding to the target subset is obtained.

[0106] The target subset includes multiple cluster groups, each of which contains one or more error samples. According to the preset similarity calculation method, the similarity between the cluster center of each cluster group and other error samples in the cluster group is calculated, and the target subset is further grouped and traversed multiple times according to the similarity and the preset group size (that is, the number of error samples contained in each group), so as to obtain the model evolution direction corresponding to the target subset.

[0107] For example, in a customer service scenario, the similarity between error results may refer to the degree of proximity of multiple incorrect answers output by the customer service model in certain features, where certain features may be customer service question category, answer style, error type, etc.

[0108] It should be noted that when traversing the target subset, grouping according to the preset grouping size and clarifying the number of erroneous samples contained in each group will help to analyze the erroneous results and the weaknesses of the model (that is, which problems are prone to errors) in more detail.

[0109] In the process of multiple group traversals, the traversal analysis results of the previous group (such as common error types, limitations of the model in processing this group of error samples, etc.) are used as guidance to help the system analyze the problems of the next group of error samples more accurately, until all error samples in the target subset are analyzed, and a comprehensive traversal analysis result corresponding to the target subset can be generated. For example, the traversal decomposition result can be an error information map of an error sample. Furthermore, based on the traversal analysis results, it is possible to determine in which aspects the model needs to be improved and summarize the direction of model evolution.

[0110] In the application scenario, according to the above steps, starting from the cluster center of the target subset, the similarity between the error results is calculated; the error results in the target subset are grouped according to the similarity and the preset grouping size, and the error results contained in each group are traversed and analyzed according to the similarity (for example, the traversal order can be in descending order of the similarity between the cluster center in the group). Through group traversal analysis, the embodiment of the present application can deeply understand in which specific types of problems the model makes mistakes, and understand the specific reasons why the model produces these error results.

[0111] Through the above-mentioned step S2224, the model training process provided by the embodiment of the present application can more accurately locate the weaknesses of the model that need to be improved, determine the direction of model evolution, and thus help the subsequent processes to iteratively adjust the model in a targeted manner. In the customer service scenario, the embodiment of the present application can help the customer model better understand and handle customer problems, reduce errors in handling common complaints, consultations, and technical support issues, and improve customer satisfaction and efficiency. The training strategy based on error result analysis provided by the embodiment of the present application helps the model learn to extract valuable information from failures and accelerate the improvement of model performance.

[0112] In an optional embodiment, in step S222, obtaining result generation experience based on the correct result includes the following method steps:

[0113] Step S2224, clustering the correct results to obtain a second clustering result;

[0114] Step S2225, selecting multiple correct samples from the second clustering results according to the preset screening condition;

[0115] Step S2226, perform cause and performance analysis on multiple correct samples to obtain result generation experience.

[0116] The above-mentioned correct results may include multiple model output results (i.e., multiple correct samples) that meet expectations or standards determined by evaluating the initial training results of the initial model. Outputting the correct results is considered as the model performing the correct behavior. For example, in a customer service scenario, the correct result may be the output answer given by the customer service model when answering a customer's question that is consistent with the expected answer.

[0117] In an exemplary implementation scenario, the above correct result may include multiple correct samples (Goodcase). A preset clustering algorithm is used to cluster multiple correct samples in the correct result to obtain a second clustering result. Specifically, the correct results are clustered according to the similarity between the correct samples. For example, in a customer service scenario, the correct answers related to the product usage direction (that is, the correct samples) are divided into one group, and the correct answers related to the return policy are divided into another group. Clustering the correct results helps to understand the performance of the model on different issues and determine in which aspects the model performs better.

[0118] The above-mentioned preset screening conditions may be rules set according to specific standards (such as representativeness of cluster centers, commonness of questions, complexity of answers, etc.). The preset screening conditions are used to select representative correct samples from each cluster group of correct results.

[0119] In the customer service scenario, based on preset screening criteria (for example, the popularity of customer questions, the accuracy of customer service answers, customer feedback scores on answers, etc.), select the correct sample that can represent the cluster group (considered to be the correct sample with the highest quality) from each cluster group of correct results, and use the correct sample as a case that can accurately reflect the performance of the model on the problem type corresponding to the cluster group.

[0120] Furthermore, the cause and performance analysis of multiple correct samples specifically includes: analyzing the reasons why multiple correct samples can achieve expectations from multiple perspectives, including the model input processing method, the model output generation logic, the model's understanding of the problem, and the model's high-quality answer generation strategy; analyzing the specific performance of the model embodied by multiple correct samples. Through the above cause and performance analysis, the result generation experience corresponding to the correct results obtained by the initial model during the initial training process is summarized.

[0121] For example, in the customer service scenario, we analyze how the customer service model understands customer questions (such as keyword matching, semantic understanding, etc.); we analyze how the customer service model generates customer service answers (such as template filling or generative answering, etc.). Through the above analysis, the system can extract the successful strategies of the customer service model when handling specific types of customer questions, and these successful strategies will be further summarized as result generation experience.

[0122] Through the above steps S2224 to S2226, the embodiment of the present application clusters, screens and deeply analyzes the successful results, and can extract the effective strategies and experiences (i.e., result generation experience) of the model in different scenarios. These effective strategies and experiences can help build higher quality training data, thereby guiding the iterative adjustment of the initial model, so that the obtained target model can guarantee high accuracy and efficiency when dealing with problems similar to the correct results. For example, in the customer service scenario, the customer service model can help the customer model provide more accurate and faster answers in future customer service conversations and improve customer satisfaction by learning and absorbing the experience of successfully answering customer questions in the past.

[0123] In an optional embodiment, in step S23, second training data is constructed based on the feedback information and the multi-role information, including the following method steps:

[0124] Step S231, constructing third training data based on the data synthesis method of context learning and feedback information;

[0125] Step S232, constructing second training data according to the third training data and the multi-role information.

[0126] In the above optional embodiment, the above feedback information is information used to guide the construction of training data determined by the system based on the result generation experience corresponding to the correct result and the model evolution direction corresponding to the wrong result. The above-mentioned data synthesis method for contextual learning may refer to a data synthesis method in which the model can understand the task context information when processing a specific task and make more accurate decisions. The data synthesis method may be a method of expanding the training set by generating or transforming data. Common data synthesis methods may involve data enhancement technology, data simulation technology, and data creation technology based on model prediction results.

[0127] For example, in the customer service scenario, in the process of constructing the third training data based on the data synthesis method of contextual learning and feedback information, the conversation between the user and the customer service can be simulated to create new conversation cases to obtain the third training data. These conversation cases can imitate the context information of the correct samples based on the feedback information, or avoid the context information of the wrong samples based on the feedback information.

[0128] It should be noted that by constructing the third training data through data synthesis based on contextual learning and feedback information, the performance of the model in specific contexts or situations can be enhanced in a targeted manner, and the model's ability to handle complex or edge situations can be improved, thereby optimizing the overall performance of the model.

[0129] In customer service scenarios, the above multi-role information can be used to set different user roles (such as ordinary users, advanced users, corporate customers, etc.), different customer service roles (such as technical support, sales representatives, customer relationship management, etc.) and different situations that may be encountered (such as returns, consultations, complaints, etc.). Multi-role information can improve the diversity of the constructed second training data, which helps the trained target model to better cope with various possible situations in practical applications.

[0130] Furthermore, after obtaining the third training data, the multi-role information is combined to further construct a training data set containing more diverse roles and situations to obtain the second training data. Specifically, the second training data can be obtained by adding dialogue cases of different roles or different situations to the third training data, and the behavior and language characteristics of different roles can be simulated to adjust the third training data to obtain the second training data.

[0131] It should be noted that by constructing the second training data based on the third training data and multi-role information, it is possible to ensure that the model is exposed to a wide range of roles and situations during the training process, thereby enhancing the adaptability and intelligence level of the model. For example, in a customer service scenario, constructing the second training data based on the third training data and multi-role information helps the customer service model better understand the needs of different users and provide more personalized and accurate services. At the same time, the second training data can also better help the customer service model cope with complex situations that different customer service roles may face, thereby improving service quality and efficiency.

[0132] Through the above-mentioned steps S231 to S232, the embodiment of the present application constructs enhanced and diversified second training data based on the feedback information obtained from the preliminary training results of the initial model, and then supports the use of the second training data for the iterative improvement process of the initial model, so that the trained target model can be more intelligent and efficient when processing specific tasks.

[0133] In an optional embodiment, in step S231, constructing third training data based on the data synthesis method of context learning and feedback information includes the following method steps:

[0134] Step S2311, determining the type of training data to be used according to the model evolution direction contained in the feedback information;

[0135] Step S2312, using the result generation experience contained in the feedback information to obtain the correct sample matching the training data type;

[0136] Step S2313, based on the data synthesis method of context learning, perform data synthesis on the correct samples that match the training data type to construct the third training data.

[0137] In the above optional embodiment, the above model evolution direction is used to determine the aspects of the model to be improved corresponding to the error results. The above training data type is the type of training data required to complete the improvement operation for the aspects of the model to be improved. For example, in a customer service scenario, suppose that when the customer service model is handling customer issues, the feedback information shows that the customer service model's explanation of product features is not accurate enough. At this time, the model evolution direction may be "enhancing the accuracy of product feature descriptions." According to this model evolution direction, the training data type to be used is determined to be "conversation data containing product features and feature explanations."

[0138] In the above optional embodiments, the above result generation experience is used to determine the model success strategy corresponding to the correct result (such as the effective strategy and experience of the model in different scenarios). Still taking the customer service scenario as an example, if the correct result in the feedback information shows that the customer service model correctly uses specific keywords and professional terms when explaining product features, then the result generation experience may be "correct use of keywords and professional terms". Based on this result generation experience, the conversation data containing these specific keywords and professional terms are filtered out from the conversation history of the customer service model, that is, the correct sample matching the training data type is obtained.

[0139] It should be noted that the correct samples matching the above training data type may be one or more. When a specified number of correct samples are obtained, the specified number of correct samples with the highest correlation with the training data type can be retrieved from the database used to store correct samples.

[0140] Furthermore, after obtaining correct samples matching the training data type, these correct samples are simulated or transformed using a data synthesis method of contextual learning to generate third training data. The third training data not only conforms to the characteristics of historical data, but also can cover more possible scenarios and changes.

[0141] For example, in a customer service scenario, if the correct sample that matches the training data type is conversation data about product feature explanations, the data synthesis method may include adjusting the specific product name, feature description, etc. in the conversation data, and / or simulating different customer questioning methods to adjust the customer questions in the conversation data. More similar but different conversation scenarios are created through data synthesis to generate training data sets for fine-tuning the customer service model.

[0142] Through the above steps S2311 to S2313, the embodiment of the present application determines the type of training data according to the direction of model evolution, ensuring the relevance and pertinence of the training data; obtains correct samples through result generation experience, and can guide model learning from a positive perspective; data synthesis based on contextual learning increases the complexity and coverage of model training, and improves the robustness and adaptability of the model when dealing with practical problems. Such a model training strategy helps the model better understand and solve complex and changeable practical problems.

[0143] In an optional embodiment, in step S24, the initial model is iteratively adjusted using the second training data to obtain a target model, including the following method steps:

[0144] Step S241, using the second training data to train the initial model to obtain a second training result;

[0145] Step S242, generating pairing data based on the second training result;

[0146] Step S243, iteratively adjust the initial model according to the paired data to obtain the target model.

[0147] In the above optional embodiment, the second training data is input into the initial model, and the model adjusts internal model parameters by learning the patterns and rules in the second training data to improve the model's prediction accuracy for new data, thereby obtaining a second training result.

[0148] The paired data may be a plurality of data pairs selected from the second training result or constructed based on the second training result. Each data pair includes data that matches under a certain condition. Further, the paired data is used to train the initial model, the current model performance is evaluated after each training, and the model parameters are adjusted according to the evaluation results, and this single training process is repeated until the model reaches the expected performance level, thereby obtaining the target model.

[0149] Through the above steps S241 to S243, the embodiment of the present application fine-tunes the model parameters of the initial model based on the automatically constructed second training data, so that the model can self-adjust according to specific scenarios and feedback information, thereby improving the accuracy and applicability of the target model.

[0150] In an optional embodiment, in step S242, generating pairing data based on the second training result includes the following method steps:

[0151] Step S2421, determining a first answer and a second answer to a specified question based on the second training result, wherein the matching degree between the specified question and the first answer is the highest, and the matching degree between the specified question and the second answer is the lowest;

[0152] Step S2422, constructing pairing data according to the first answer and the second answer.

[0153] The above-mentioned specified question may be a user question processed by the model in the second training data (may be included in the second training data). The above-mentioned first answer is the most expected and accurate answer generated by the model for the specified question. The above-mentioned second answer is the least expected and inaccurate answer generated by the model for the specified question. For example, the specified question may be "How do I reset my account password?" Correspondingly, the first answer may include detailed password reset steps and security suggestions; the second answer may answer how to change the username and completely deviate from the topic of password reset.

[0154] In the above optional embodiment, the output of the model on the second training data is evaluated to identify the best answer (first answer) and the worst answer (second answer) for each question. Here, the quality of the answer is determined based on the degree of match between the answer output by the model and the specified question. The higher the degree of match, the more accurate and relevant the answer is to the specified question; the lower the degree of match, the more likely the answer is to deviate from the specified question or is completely irrelevant to the specified question.

[0155] Furthermore, the first answer and the second answer are combined to obtain paired data corresponding to the specified question. Based on this, when the paired data is used for model training, the model's ability to identify correct answers and avoid incorrect answers can be enhanced.

[0156] Through the above steps S2421 to S2422, the embodiment of the present application helps the model understand and learn what kind of answers are high-quality and what kind of answers are low-quality by constructing paired data for iterative adjustment of the initial model. This model comparison learning method can not only strengthen the model's memory of the correct answers, but also allow the model to learn to avoid generating wrong answers, thereby improving the accuracy and stability of the model as a whole.

[0157] For example, in customer service scenarios, the above-mentioned comparative learning method can help the customer service model understand customer questions more accurately and give the most appropriate and professional customer service answers, reducing incorrect answers output due to misunderstandings or knowledge blind spots, and improving customer satisfaction and interactive experience. At the same time, through continuous iteration and optimization, the customer service model can gradually reduce errors and improve its responsiveness and answer quality to various customer questions.

[0158] In an optional embodiment, in step S243, the initial model is iteratively adjusted according to the paired data to obtain the target model, including the following method steps:

[0159] Step S2431, training the initial model according to the paired data, and determining a first loss and a second loss, wherein the first loss is a model fine-tuning loss, and the second loss is a preference ranking optimization loss;

[0160] Step S2432, iteratively adjust the initial model based on the first loss and the second loss to obtain the target model.

[0161] The first loss (SFT Loss) can be a loss calculated based on the difference between the model's prediction result and the actual correct answer. The first loss is used to guide the fine-tuning of the parameters of the initial model to ensure that the prediction result of the trained model is as close to the actual correct answer as possible.

[0162] The second loss (PRO Loss) is a loss calculated using a specific loss function to evaluate and optimize the model's ability to rank multiple possible answers. The second loss is used to ensure that the trained model can not only generate multiple correct candidate answers, but also prioritize multiple candidate answers based on user preferences or context, so as to select the correct answer to be used.

[0163] In the above optional embodiment, when training the initial model, the model will calculate the first loss based on the difference between the prediction result and the first answer in the paired data, and calculate the second loss based on the matching degree between the model's predicted ranking and the actual user preference or context. Further, through continuous iteration (multiple training and adjustment), the model will fine-tune the model parameters based on the first loss and the second loss in each iteration to reduce the loss values ​​of the first loss and the second loss, thereby obtaining the target model.

[0164] Through the above steps S2431 to S2432, the embodiment of the present application ensures the accuracy of the model when answering specific questions by introducing the optimization of the model fine-tuning loss, thereby reducing the possibility of misunderstanding or wrong answers; by introducing the preference sorting optimization loss, the model can better understand the user's preferences, provide answers that are closer to user needs, and enhance user interaction satisfaction; by learning the context and user preferences, the model can quickly provide the most relevant and effective answers, reduce user waiting time, and improve the overall efficiency of customer service.

[0165] In an exemplary application scenario, the following is provided: Figure 3 A model training process is shown in Figure 1. Figure 3As shown, the model training process includes a preliminary training stage. In the preliminary training stage, the initial model is preliminarily trained using the evaluation set to obtain the first training result. The first training result at least includes the inference results generated by the initial model for multiple evaluation samples in the evaluation set. Furthermore, the first training result is evaluated using a large language model. During the evaluation process, in order to increase the richness and accuracy of the evaluation results, each inference result in the first training result can be scored from different evaluation dimensions, and the inference results with a score of more than 8 points are defined as excellent cases, thereby producing the evaluation results. The evaluation result is used as the input for the next stage.

[0166] Still like Figure 3 As shown, the model training process also includes a result analysis phase, during which feedback information will be generated, including: multiple correct samples determined by the evaluation results ( Figure 3 The results of summarizing the samples G1 to Gn shown in FIG. 1 and the multiple error samples ( Figure 3 The results obtained by analyzing the samples B1 to Bn shown in FIG. 1 and FIG. 2 are used to generate experience. In addition, in the process of analyzing the correct results and the wrong results, the clustering method and the grouping traversal method can also be used.

[0167] Still like Figure 3 As shown, the model training process also includes a training data construction stage. In the training data construction stage, the required training data type is determined based on the model evolution direction, and three correct samples with the highest correlation with the training data type are retrieved from the database, that is, the correct samples are matched with the training data type determined by the model evolution direction, and the third training data is further obtained through data synthesis; the third training data and multi-role information are combined to generate the second training data.

[0168] Still like Figure 3 As shown, the model training process also includes a model fine-tuning stage. In the model fine-tuning stage, paired data is constructed based on the second training data, and the paired data includes the best answer and the worst answer corresponding to a certain question. Further, the model is iteratively adjusted based on the paired data, and the iterative adjustment process includes multiple rounds of model parameter adjustment based on the first loss and the second loss (such as gradient calculation and back propagation based on the total loss), thereby obtaining the target model.

[0169] Therefore, if Figure 3 The model training process shown is a process that supports model self-evolution. In application scenarios, technicians no longer need to spend a lot of time checking and analyzing bad cases one by one. They can obtain information about the direction of model evolution, complete automated training data construction, and achieve efficient training of the target model.

[0170] The embodiment of the present application provides a self-evolving model training strategy, in which feedback information is generated by analyzing the initial training results of the model, thereby guiding the construction process of the training data and training to obtain the target model, thereby improving the model training efficiency and model training effect. The entire model training process can be continuously iterated to continuously improve the model effect, reducing the time cost and labor cost required in the model optimization process.

[0171] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0172] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0173] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0174] In the aforementioned operating environment, the present application also provides Figure 4 A data processing method is shown. Figure 4 is a flow chart of a data processing method according to an embodiment of the present application, such as Figure 4 As shown, the data processing method includes:

[0175] Step S41, obtaining a target question to be answered;

[0176] Step S42, using the target model to analyze the target question and generate a target answer;

[0177] Among them, the target model is obtained by iteratively adjusting the initial model with the target training data. The target training data is constructed based on feedback information and multi-role information. The multi-role information is used to set multiple different roles in the process of training data construction. The feedback information is obtained by evaluating the initial training results from multiple evaluation dimensions. The initial training results are obtained by training the initial model with the initial training data.

[0178] The target question to be answered is a question input by a user in an application scenario, and the target question is used as a problem to be processed by the target model. After the target model is trained using the aforementioned model training method, the target model will analyze the target question and generate a target answer.

[0179] The specific training method of the target model in the embodiment of the present application can refer to the explanation in the previous text and will not be repeated here. Since the aforementioned model training method can achieve the purpose of automatically constructing model fine-tuning training data and reducing the dependence of the model training process on manual work, it achieves the technical effect of improving the automation and comprehensiveness of the training data construction process, improving the training efficiency and training effect of the target model, and solving the technical problem of low model training efficiency and poor effect caused by the high degree of manual dependence on the construction of the training data set. The use of the above-mentioned target model in the embodiment of the present application can give a target answer with high accuracy and strong pertinence, thereby improving user satisfaction.

[0180] In the aforementioned operating environment, this application provides the following Figure 5 A data processing method is shown. Figure 5 is a flow chart of a data processing method according to an embodiment of the present application, such as Figure 5 As shown, the data processing method includes:

[0181] Step S51, obtaining customer service questions to be answered;

[0182] Step S52, using the target customer service model to analyze the customer service question and generate a customer service answer;

[0183] Among them, the target customer service model is obtained by iteratively adjusting the initial customer service model with the target customer service training data. The target customer service training data is constructed based on customer service feedback information and multi-role information. The multi-role information is used to set multiple different customer service roles in the process of constructing the customer service training data. The customer service feedback information is obtained by evaluating the initial customer service training results from multiple evaluation dimensions. The initial customer service training results are obtained by training the initial customer service model with the initial customer service training data.

[0184] The customer service questions to be answered are questions input by customers in the customer service application scenario, and the customer service questions are questions to be processed by the target customer service model. After the target customer service model is trained for the customer service application scenario using the aforementioned model training method, the target customer service model will analyze the customer service questions and generate customer service answers.

[0185] The specific training method of the target customer service model in the embodiment of the present application can refer to the explanation in the previous text and will not be repeated here. Since the aforementioned model training method can achieve the purpose of automatically constructing model fine-tuning training data and reducing the dependence of the model training process on manual work, it achieves the technical effect of improving the automation and comprehensiveness of the training data construction process, improving the training efficiency and training effect of the target model, and solving the technical problem of low model training efficiency and poor effect caused by the high degree of manual dependence on the construction of the training data set. The use of the above-mentioned target customer service model in the embodiment of the present application can give customer service answers with higher accuracy and stronger pertinence, and improve customer satisfaction with customer service.

[0186] In the aforementioned operating environment, this application provides the following Figure 6 A data processing method is shown. Figure 6 is a flow chart of a data processing method according to an embodiment of the present application, such as Figure 6 As shown, the data processing method includes:

[0187] Step S61, obtaining a data processing request through a first application programming interface, wherein the request data carried in the data processing request includes: a target question to be answered;

[0188] Step S62, returning a data processing response through a second application programming interface, wherein the response data carried in the data processing response includes: a target answer, the target answer is obtained by analyzing the target question with a target model, the target model is obtained by iteratively adjusting the initial model with target training data, the target training data is constructed based on feedback information and multi-role information, the multi-role information is used to set multiple different roles in the process of training data construction, the feedback information is obtained by evaluating the initial training results from multiple evaluation dimensions, and the initial training results are obtained by training the initial model with the initial training data.

[0189] According to the above method steps, a method for implementing a data processing cloud service is provided, and the method runs on a cloud server. The cloud server obtains a data processing request issued by a service caller through a first application programming interface, executes a data processing flow based on a target question to be answered carried in the data processing request, and obtains a target answer. Furthermore, the cloud server returns a data processing response to the service caller through a second application programming interface to provide the target answer to the service caller.

[0190] The first application programming interface and the second application programming interface may be the same application programming interface or different application programming interfaces. In an optional embodiment, the interface parameters in the first application programming interface and the second application programming interface may include but are not limited to: interface global identifier, interface signature key, interface timestamp, interface request identifier, system call credential identifier, etc. The first application programming interface may use the acquisition method or the submission method as the interface request method to obtain the file processing request. The second application programming interface may feedback the file processing response in JSON format.

[0191] The specific training method of the target model in the embodiment of the present application can refer to the explanation in the previous text and will not be repeated here. Since the aforementioned model training method can achieve the purpose of automatically constructing model fine-tuning training data and reducing the dependence of the model training process on manual work, it achieves the technical effect of improving the automation and comprehensiveness of the training data construction process, improving the training efficiency and training effect of the target model, and solving the technical problem of low model training efficiency and poor effect caused by the high degree of manual dependence on the construction of the training data set. The use of the above-mentioned target model in the embodiment of the present application can give a target answer with high accuracy and strong pertinence, thereby improving user satisfaction.

[0192] In the aforementioned operating environment, this application provides the following Figure 7 A data processing method is shown. Figure 7 is a flow chart of a data processing method according to an embodiment of the present application, such as Figure 7 As shown, the data processing method includes:

[0193] Step S71, obtaining the currently input data processing dialogue request, wherein the request data carried in the data processing dialogue request includes: the target question to be answered;

[0194] Step S72, in response to the data processing dialogue request, returning a data processing dialogue reply, wherein the information carried in the data processing dialogue reply includes: a target answer, the target answer is obtained by analyzing the target question using a target model, the target model is obtained by iteratively adjusting the initial model using target training data, the target training data is constructed based on feedback information and multi-role information, the multi-role information is used to set multiple different roles in the training data construction process, the feedback information is obtained by evaluating the initial training result from multiple evaluation dimensions, and the initial training result is obtained by training the initial model using the initial training data;

[0195] Step S73, displaying the target answer in the graphical user interface.

[0196] According to the above method steps, a visualization scheme for data processing functions is provided. The terminal device provides a graphical user interface, in which at least one data processing scenario is displayed. The display content of the graphical user interface also includes an input component (such as a text input box, a voice input control, etc.) and a display component (such as a text display window). The user inputs a data processing dialogue request through the input component to specify the target question to be answered in the data processing task. After detecting the user's input behavior, the data processing process is executed based on the target question to be answered to obtain the target answer, and further, the target answer is displayed through the display component in the graphical user interface.

[0197] The specific training method of the target model in the embodiment of the present application can refer to the explanation in the previous text and will not be repeated here. Since the aforementioned model training method can achieve the purpose of automatically constructing model fine-tuning training data and reducing the dependence of the model training process on manual work, it achieves the technical effect of improving the automation and comprehensiveness of the training data construction process, improving the training efficiency and training effect of the target model, and solving the technical problem of low model training efficiency and poor effect caused by the high degree of manual dependence on the construction of the training data set. The use of the above-mentioned target model in the embodiment of the present application can give a target answer with high accuracy and strong pertinence, thereby improving user satisfaction.

[0198] According to an embodiment of the present application, a device embodiment for implementing the above-mentioned model training method is also provided. Figure 8 is a structural diagram of a model training device according to an embodiment of the present application, such as Figure 8As shown, the model training device includes: a training module 801, which is used to train the initial model with the first training data to obtain a first training result; an evaluation module 802, which is used to evaluate the first training result from multiple evaluation dimensions to obtain feedback information; a construction module 803, which is used to construct the second training data based on the feedback information and the multi-role information, wherein the multi-role information is used to set a plurality of different roles in the process of training data construction; an iteration module 804, which is used to iteratively adjust the initial model with the second training data to obtain a target model.

[0199] Optionally, the above-mentioned evaluation module 802 is also used to: evaluate the first training result from multiple evaluation dimensions to obtain evaluation results, wherein the evaluation results include: correct results and incorrect results; obtain result generation experience based on the correct results, and determine the model evolution direction based on the incorrect results; determine feedback information based on the result generation experience and the model evolution direction.

[0200] Optionally, the above-mentioned evaluation module 802 is also used to: determine multiple evaluation indicators from multiple evaluation dimensions, wherein the multiple evaluation indicators include: evaluation indicators for the correctness of the evaluation results, evaluation indicators for the integrity of the evaluation results, evaluation indicators for the format of the evaluation results, and evaluation indicators for the instruction compliance of the evaluation results; based on at least some of the multiple evaluation indicators, evaluate the first training result to obtain an evaluation result.

[0201] Optionally, the evaluation module 802 is further used to: cluster the error results to obtain a first clustering result; determine the target subset based on the first clustering result; and determine the model evolution direction corresponding to the target subset by group traversal starting from the cluster center of the target subset.

[0202] Optionally, the above-mentioned evaluation module 802 is also used to: start from the cluster center of the target subset, perform multiple group traversals on the target subset based on the similarity between the error results contained in the target subset and the preset group size, and set the traversal analysis results of the previous group as the traversal analysis input of the next group during the group traversal process, until all groups are traversed and the model evolution direction corresponding to the target subset is obtained.

[0203] Optionally, the evaluation module 802 is further used to: cluster the correct results to obtain a second clustering result; select multiple correct samples from the second clustering result according to preset screening conditions; perform cause and performance analysis on the multiple correct samples to obtain result generation experience.

[0204] Optionally, the construction module 803 is further used to: construct third training data based on the data synthesis method of context learning and feedback information; and construct second training data based on the third training data and multi-role information.

[0205] Optionally, the above-mentioned construction module 803 is also used to: determine the type of training data to be used based on the model evolution direction contained in the feedback information; use the result generation experience contained in the feedback information to obtain correct samples that match the training data type; based on the data synthesis method of context learning, perform data synthesis on the correct samples that match the training data type to construct the third training data.

[0206] Optionally, the iteration module 804 is further used to: train the initial model using the second training data to obtain a second training result; generate pairing data based on the second training result; and iteratively adjust the initial model according to the pairing data to obtain a target model.

[0207] Optionally, the above-mentioned iteration module 804 is also used to: determine the first answer and the second answer of the specified question based on the second training result, wherein the matching degree between the specified question and the first answer is the highest, and the matching degree between the specified question and the second answer is the lowest; and construct pairing data based on the first answer and the second answer.

[0208] Optionally, the above-mentioned iterative module 804 is also used to: train the initial model based on the paired data, determine the first loss and the second loss, wherein the first loss is the model fine-tuning loss, and the second loss is the preference sorting optimization loss; iteratively adjust the initial model based on the first loss and the second loss to obtain the target model.

[0209] It should be noted here that the above-mentioned training module 801, evaluation module 802, construction module 803 and iteration module 804 correspond to steps S21 to S24 in the embodiment, and the four modules are the same as the instances and application scenarios implemented by the corresponding steps, but are not limited to the contents disclosed in the aforementioned embodiment.

[0210] According to an embodiment of the present application, a device embodiment for implementing the data processing method in the above embodiment is also provided. Fig. 9 is a structural diagram of a data processing device according to an embodiment of the present application, such as Fig. 9 As shown, the device includes: an acquisition module 901, used to obtain a target problem to be answered; an analysis module 902, used to analyze the target problem using a target model to generate a target answer; wherein the target model is obtained by iteratively adjusting the initial model using target training data, the target training data is constructed based on feedback information and multi-role information, the multi-role information is used to set multiple different roles in the process of training data construction, the feedback information is obtained by evaluating the initial training results from multiple evaluation dimensions, and the initial training results are obtained by training the initial model using the initial training data.

[0211] It should be noted here that the above-mentioned acquisition module 901 and analysis module 902 correspond to the aforementioned steps S41 to S42, and the instances and application scenarios implemented by the two modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned embodiments.

[0212] According to an embodiment of the present application, another data processing device is provided for a customer service application scenario, and the data processing device includes: an acquisition module, used to obtain customer service questions to be answered; an analysis module, used to analyze customer service questions using a target customer service model, and generate customer service answers; wherein the target customer service model is obtained by iteratively adjusting the initial customer service model using target customer service training data, the target customer service training data is constructed based on customer service feedback information and multi-role information, the multi-role information is used to set multiple different customer service roles in the process of constructing the customer service training data, the customer service feedback information is obtained by evaluating the initial customer service training results from multiple evaluation dimensions, and the initial customer service training results are obtained by training the initial customer service model using the initial customer service training data.

[0213] It should be noted here that the above-mentioned acquisition module and analysis module correspond to the aforementioned steps S51 to S52, and the instances and application scenarios implemented by the two modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned embodiments.

[0214] According to an embodiment of the present application, for a cloud service interface application scenario, another data processing device is also provided, which includes: a request module, used to obtain a data processing request through a first application programming interface, wherein the request data carried in the data processing request includes: a target question to be answered; a response module, used to return a data processing response through a second application programming interface, wherein the response data carried in the data processing response includes: a target answer, the target answer is obtained by analyzing the target question using a target model, the target model is obtained by iteratively adjusting the initial model using target training data, the target training data is constructed based on feedback information and multi-role information, the multi-role information is used to set multiple different roles in the process of training data construction, the feedback information is obtained by evaluating the initial training results from multiple evaluation dimensions, and the initial training results are obtained by training the initial model using the initial training data.

[0215] It should be noted here that the above-mentioned request module and response module correspond to the aforementioned steps S61 to S62, and the instances and application scenarios implemented by the two modules and the corresponding steps are the same, but are not limited to the contents disclosed in the above-mentioned embodiments.

[0216] According to an embodiment of the present application, another data processing device is also provided for a human-computer interaction application scenario, and the data processing device includes: an acquisition module, which is used to obtain a currently input data processing dialogue request, wherein the request data carried in the data processing dialogue request includes: a target question to be answered; a reply module, which is used to respond to the data processing dialogue request and return a data processing dialogue reply, wherein the information carried in the data processing dialogue reply includes: a target answer, the target answer is obtained by analyzing the target question using a target model, the target model is obtained by iteratively adjusting the initial model using target training data, the target training data is constructed based on feedback information and multi-role information, the multi-role information is used to set multiple different roles in the training data construction process, the feedback information is obtained by evaluating the initial training results from multiple evaluation dimensions, and the initial training results are obtained by training the initial model using the initial training data; a display module, which is used to display the target answer in a graphical user interface.

[0217] It should be noted here that the above-mentioned acquisition module, reply module and display module correspond to the aforementioned steps S71 to S73, and the three modules have the same instances and application scenarios implemented by the corresponding steps, but are not limited to the contents disclosed in the above-mentioned embodiments.

[0218] It should be noted that the above modules or units may be hardware components or software components stored in a memory and processed by one or more processors, and the above modules may also be run in a computer terminal as a part of the device.

[0219] It should be noted that the preferred implementation of this embodiment can refer to the relevant descriptions in the aforementioned embodiments, which will not be repeated here.

[0220] According to an embodiment of the present application, an electronic device is also provided, which can be any terminal device in a computer terminal group. Optionally, in this embodiment, the electronic device can also be replaced by a terminal device such as a mobile terminal.

[0221] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.

[0222] In this embodiment, the above-mentioned electronic device can execute the program code of the following steps in the model training method: use the first training data to train the initial model to obtain a first training result; evaluate the first training result from multiple evaluation dimensions to obtain feedback information; construct the second training data based on the feedback information and the multi-role information, wherein the multi-role information is used to set multiple different roles in the training data construction process; use the second training data to iteratively adjust the initial model to obtain the target model.

[0223] In this embodiment, the above-mentioned electronic device can execute the program code of the following steps in the data processing method: obtaining a target problem to be answered; using a target model to analyze the target problem and generate a target answer; wherein the target model is obtained by iteratively adjusting the initial model using target training data, the target training data is constructed based on feedback information and multi-role information, the multi-role information is used to set multiple different roles in the training data construction process, the feedback information is obtained after evaluating the initial training results from multiple evaluation dimensions, and the initial training results are obtained after training the initial model using the initial training data.

[0224] Optionally, Fig.10 is a structural block diagram of an electronic device according to Embodiment 5 of the present application, such as Fig.10 As shown, the electronic device 100 may include: one or more (only one is shown in the figure) processors 1002, a memory 1004, a storage controller 1006, and a peripheral interface 1008, wherein the peripheral interface 1008 is connected to a radio frequency module, an audio module, and a display.

[0225] Among them, the memory 1004 can be used to store software programs and modules, such as the model training method, data processing method and program instructions / modules corresponding to the related devices in the embodiments of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, realizing the above-mentioned model training method or data processing method. The memory 1004 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1004 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the electronic device 100 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network and a combination thereof.

[0226] The processor 1002 can call the information and application stored in the memory through the transmission device to perform the following steps: use the first training data to train the initial model to obtain a first training result; evaluate the first training result from multiple evaluation dimensions to obtain feedback information; construct the second training data based on the feedback information and the multi-role information, wherein the multi-role information is used to set multiple different roles in the training data construction process; use the second training data to iteratively adjust the initial model to obtain the target model.

[0227] Processor 1002 can call the information and application programs stored in the memory through the transmission device to perform the following steps: obtain the target problem to be answered; use the target model to analyze the target problem and generate a target answer; wherein the target model is obtained by iteratively adjusting the initial model using the target training data, the target training data is constructed based on the feedback information and the multi-role information, the multi-role information is used to set multiple different roles in the training data construction process, the feedback information is obtained by evaluating the initial training results from multiple evaluation dimensions, and the initial training results are obtained by training the initial model using the initial training data.

[0228] By adopting the embodiment of the present application, a scheme of an electronic device for implementing the above-mentioned model training method or data processing method is provided. The first training result is obtained by training the initial model with the first training data; the first training result is evaluated from multiple evaluation dimensions to obtain feedback information; the second training data is further constructed based on the feedback information and the multi-role information, and the multi-role information is used to set multiple different roles in the training data construction process; and the initial model is iteratively adjusted using the second training data to obtain the target model. Based on this, the present application evaluates the pre-trained model, and comprehensively considers the multiple different roles set in the training data construction process, and automatically constructs the second training data for model fine-tuning iteration. Since the feedback information obtained from the above evaluation can characterize the training effect of the first training result of the initial model from multiple evaluation dimensions, and the automatically constructed second training data is guided by the above feedback information and multi-role information, the embodiment of the present application can reduce the dependence of the training data construction process on manual labor, the construction basis of the training data is more comprehensive, and the training mechanism of the target model is optimized, so as to achieve the purpose of automatically constructing the model fine-tuning training data and reducing the dependence of the model training process on manual labor, thereby achieving the technical effect of improving the automation degree and comprehensiveness of the training data construction process, and improving the training efficiency and training effect of the target model, thereby solving the technical problem of low model training efficiency and poor effect caused by the high degree of manual dependence on the construction of the training data set.

[0229] It can be understood by those skilled in the art that Fig.10 The structure shown is for illustration only, and the electronic device may also be a terminal device such as a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, and a mobile Internet device (Mobile Internet Devices, MID). Fig.10 The structure of the electronic device is not limited. Fig.10 More or fewer components (such as network interfaces, display devices, etc.) shown in, or with Fig.10 Different configurations shown.

[0230] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which can include: a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.

[0231] According to an embodiment of the present application, a computer-readable storage medium is also provided. Optionally, in this embodiment, the above storage medium can be used to store the program code executed by the model training method or data processing method provided in the above embodiment.

[0232] Optionally, in this embodiment, the above storage medium may be located in any computer terminal in a computer terminal group in a computer network, or in any mobile terminal in a mobile terminal group.

[0233] Optionally, in this embodiment, the computer-readable storage medium is configured to store program codes for executing the following steps: training the initial model using the first training data to obtain a first training result; evaluating the first training result from multiple evaluation dimensions to obtain feedback information; constructing the second training data based on the feedback information and the multi-role information, wherein the multi-role information is used to set multiple different roles in the training data construction process; and iteratively adjusting the initial model using the second training data to obtain the target model.

[0234] Optionally, in this embodiment, a computer-readable storage medium is configured to store program code for executing the following steps: obtaining a target problem to be answered; analyzing the target problem using a target model to generate a target answer; wherein the target model is obtained by iteratively adjusting the initial model using target training data, the target training data is constructed based on feedback information and multi-role information, the multi-role information is used to set multiple different roles in the training data construction process, the feedback information is obtained by evaluating the initial training results from multiple evaluation dimensions, and the initial training results are obtained by training the initial model using the initial training data.

[0235] The embodiment of the present application also provides a computer program product. Optionally, in this embodiment, the computer program product may include a computer program, and when the computer program is executed by a processor, the model training method or data processing method provided in the embodiment is implemented.

[0236] The embodiments of the present application also provide a computer program product. Optionally, the computer program product may include a non-volatile computer-readable storage medium, which may be used to store a computer program, and when the computer program is executed by a processor, the method provided in the embodiments is implemented.

[0237] The embodiment of the present application further provides a computer program. Optionally, in this embodiment, when the computer program is executed by a processor, the method provided in the above embodiment is implemented.

[0238] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0239] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0240] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0241] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0242] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, ROM, RAM, mobile hard disks, magnetic disks or optical disks.

[0243] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A model training method, characterized in that: include: Using the first training data to train the initial model to obtain a first training result; Evaluate the first training result from multiple evaluation dimensions to obtain feedback information; constructing second training data based on the feedback information and the multi-role information, wherein the multi-role information is used to set a plurality of different roles in the training data construction process; The initial model is iteratively adjusted using the second training data to obtain a target model.

2. The model training method according to claim 1, characterized in that: Evaluating the first training result from the multiple evaluation dimensions to obtain the feedback information includes: Evaluating the first training result from the multiple evaluation dimensions to obtain evaluation results, wherein the evaluation results include: correct results and incorrect results; Obtaining result generation experience based on the correct result, and determining a model evolution direction based on the incorrect result; The feedback information is determined according to the result generation experience and the model evolution direction.

3. The model training method according to claim 2, characterized in that: Evaluating the first training result from the multiple evaluation dimensions to obtain the evaluation result includes: Determining a plurality of evaluation indicators from the plurality of evaluation dimensions, wherein the plurality of evaluation indicators include at least one of the following: an evaluation indicator of the correctness of the evaluation result, an evaluation indicator of the integrity of the evaluation result, an evaluation indicator of the format of the evaluation result, and an evaluation indicator of the instruction compliance of the evaluation result; Based on at least some of the multiple evaluation indicators, the first training result is evaluated to obtain the evaluation result.

4. The model training method according to claim 2, characterized in that: Determining the model evolution direction based on the error result includes: Clustering the error results to obtain a first clustering result; Determining a target subset according to the first clustering result; Starting from the cluster center of the target subset, a group traversal method is used to determine the model evolution direction corresponding to the target subset.

5. The model training method according to claim 4, characterized in that: Starting from the cluster center of the target subset, determining the model evolution direction corresponding to the target subset by using the group traversal method includes: Starting from the cluster center of the target subset, the target subset is traversed in groups multiple times according to the similarity between the error results contained in the target subset and the preset group size, and during the group traversal process, the traversal analysis result of the previous group is set as the traversal analysis input of the next group until all groups are traversed, thereby obtaining the model evolution direction corresponding to the target subset.

6. The model training method according to claim 2, characterized in that: Acquiring the result generation experience based on the correct result includes: Clustering the correct results to obtain a second clustering result; Selecting a plurality of correct samples from the second clustering results according to a preset screening condition; Performing cause and performance analysis on the multiple correct samples to obtain the result generation experience.

7. The model training method according to claim 2, characterized in that: Constructing the second training data based on the feedback information and the multi-role information includes: Constructing third training data based on the data synthesis method of context learning and the feedback information; The second training data is constructed according to the third training data and the multi-role information.

8. The model training method according to claim 7, characterized in that: Constructing the third training data based on the context-based learning data synthesis method and the feedback information includes: Determining the type of training data to be used according to the model evolution direction contained in the feedback information; Using the result generation experience contained in the feedback information to obtain a correct sample matching the training data type; Based on the data synthesis method of context learning, data synthesis is performed on the correct samples matching the training data type to construct the third training data.

9. The model training method according to claim 1, characterized in that: Iteratively adjusting the initial model using the second training data to generate the target model includes: Using the second training data to train the initial model to obtain a second training result; generating pairing data based on the second training result; The initial model is iteratively adjusted according to the pairing data to generate the target model.

10. The model training method according to claim 9, characterized in that: Generating the pairing data based on the second training result includes: Determine a first answer and a second answer to a specified question based on the second training result, wherein the matching degree between the specified question and the first answer is the highest, and the matching degree between the specified question and the second answer is the lowest; The pairing data is constructed according to the first answer and the second answer.

11. The model training method according to claim 9, characterized in that: Iteratively adjusting the initial model according to the paired data to generate the target model includes: Training the initial model according to the paired data to determine a first loss and a second loss, wherein the first loss is a model fine-tuning loss, and the second loss is a preference ranking optimization loss; The initial model is iteratively adjusted based on the first loss and the second loss to generate the target model.

12. A data processing method, characterized in that: include: Get the target question to be answered; Using the target model to analyze the target problem and generate a target answer; Among them, the target model is obtained by iteratively adjusting the initial model using target training data, the target training data is constructed based on feedback information and multi-role information, the multi-role information is used to set multiple different roles in the process of training data construction, the feedback information is obtained by evaluating the initial training results from multiple evaluation dimensions, and the initial training results are obtained by training the initial model using the initial training data.

13. A data processing method, characterized in that: include: Get answers to customer service questions; Analyze the customer service problem using the target customer service model to generate a customer service answer; Among them, the target customer service model is obtained by iteratively adjusting the initial customer service model using target customer service training data, the target customer service training data is constructed based on customer service feedback information and multi-role information, the multi-role information is used to set multiple different customer service roles in the process of constructing the customer service training data, the customer service feedback information is obtained by evaluating the initial customer service training results from multiple evaluation dimensions, and the initial customer service training results are obtained by training the initial customer service model with the initial customer service training data.

14. A data processing method, characterized in that: include: Obtaining a data processing request through a first application programming interface, wherein the request data carried in the data processing request includes: a target question to be answered; A data processing response is returned through a second application programming interface, wherein the response data carried in the data processing response includes: a target answer, wherein the target answer is obtained by analyzing the target question with a target model, wherein the target model is obtained by iteratively adjusting the initial model with target training data, wherein the target training data is constructed based on feedback information and multi-role information, wherein the multi-role information is used to set multiple different roles in the process of constructing training data, wherein the feedback information is obtained by evaluating the initial training results from multiple evaluation dimensions, and wherein the initial training results are obtained by training the initial model with the initial training data.

15. A data processing method, characterized in that: include: Acquire a currently input data processing dialogue request, wherein the request data carried in the data processing dialogue request includes: a target question to be answered; In response to the data processing dialogue request, a data processing dialogue reply is returned, wherein the information carried in the data processing dialogue reply includes: a target answer, the target answer is obtained by analyzing the target question using a target model, the target model is obtained by iteratively adjusting the initial model using target training data, the target training data is constructed based on feedback information and multi-role information, the multi-role information is used to set multiple different roles in the training data construction process, the feedback information is obtained by evaluating the initial training result from multiple evaluation dimensions, and the initial training result is obtained by training the initial model using the initial training data; The target answer is displayed in a graphical user interface.

16. An electronic device, characterized in that: include: A memory storing an executable program; A processor, used to run the program, wherein when the program is run, it executes the model training method described in any one of claims 1 to 11 or the data processing method described in any one of claims 12 to 15.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein, when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the model training method described in any one of claims 1 to 11 or the data processing method described in any one of claims 12 to 15.

18. A computer program product, characterized in that It includes a computer program, which, when executed by a processor, implements the model training method described in any one of claims 1 to 11 or executes the data processing method described in any one of claims 12 to 15.

Images