Model optimization method, data processing method, device, equipment and program product

Abstract

The invention provides a model optimization method and device, a data processing method and device, equipment and a program product. The model optimization method comprises the following steps: when an ith processing result obtained by processing first task data by a task model is obtained, inputting the first task data and the ith processing result into an evaluation model for evaluating the processing result of the task model to obtain an ith evaluation result corresponding to the ith processing result; the first task data comprises at least one of text data, image data and voice data, and the task model is used for executing at least one of a text processing task, an image processing task and a voice processing task; and when it is determined that the ith processing result needs to be optimized based on the ith evaluation result, inputting the ith evaluation result into the task model, so that the task model processes the first task data based on the ith evaluation result to obtain an (i + 1) th processing result.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model optimization method, data processing method, device, equipment and program product. Background Art

[0002] Large language models have shown excellent performance in a variety of data processing tasks due to their excellent understanding, data processing and learning capabilities. However, large language models do not have the ability to meet user needs in all scenarios, and in some cases there are output errors.

[0003] In order to make the large language model have the working ability that meets user needs, the traditional solution is to fine-tune the large language model. However, some large language models are closed or have limited access, and it is impossible to directly fine-tune the model parameters. Summary of the invention

[0004] In view of this, the embodiments of the present application are committed to providing a model optimization method, data processing method, device, equipment and program product, which can enable the task model to optimize the data processing results based on the evaluation results of the evaluation model.

[0005] According to a first aspect of an embodiment of the present application, a model optimization method is provided, comprising:

[0006] In the case of obtaining an i-th processing result obtained by processing first task data by a task model, inputting the first task data and the i-th processing result into an evaluation model for evaluating the processing result of the task model, to obtain an i-th evaluation result corresponding to the i-th processing result; the first task data includes at least one of text data, image data, and voice data, and the task model is used to perform at least one of a text processing task, an image processing task, and a voice processing task;

[0007] When it is determined based on the i-th evaluation result that the i-th processing result needs to be optimized, the i-th evaluation result is input into the task model so that the task model processes the first task data based on the i-th evaluation result to obtain the i+1th processing result; wherein i is a positive integer.

[0008] Optionally, the method further includes:

[0009] In the case where it is determined based on the kth evaluation result output by the evaluation model that the kth processing result output by the task model meets the preset processing result requirements, the first task data and the kth processing result are used to form a training data pair; k is a positive integer;

[0010] The task model is optimized and trained using the training data pairs.

[0011] Optionally, the training process of the evaluation model includes:

[0012] Acquire historical data pairs and evaluation result labels corresponding to each historical data pair, wherein the historical data pairs include historical data and data processing results obtained by the task model processing the historical data;

[0013] Based on the historical data pairs and the evaluation result labels corresponding to each historical data pair, the evaluation model is trained to generate evaluation results.

[0014] Optionally, the training process of the evaluation model further includes:

[0015] Inputting the first historical task data and the first data processing result corresponding to the first historical task data into the evaluation model to obtain a first historical evaluation result;

[0016] inputting the first historical evaluation result into the task model so that the task model generates a second data processing result corresponding to the first historical task data based on the first historical evaluation result;

[0017] Determining a score for the first historical evaluation result based on the first data processing result and the second data processing result;

[0018] With the goal of improving the score, the parameters of the evaluation model are optimized.

[0019] Optionally, determining a score for the first historical evaluation result based on the first data processing result and the second data processing result includes:

[0020] Scoring the first data processing result to obtain a first score, and scoring the second data processing result to obtain a second score;

[0021] A difference between the first score and the second score is calculated as a score for the first historical evaluation result.

[0022] Optionally, scoring the first data processing result to obtain a first score, and scoring the second data processing result to obtain a second score, includes:

[0023] Inputting the first historical task data and the first data processing result into a pre-trained reward model to obtain a first score corresponding to the first data processing result;

[0024] And, the first historical task data and the second data processing result are input into the reward model to obtain a second score corresponding to the second data processing result.

[0025] According to a second aspect of an embodiment of the present application, a data processing method is provided, including:

[0026] Inputting the data to be processed into a pre-trained task model so that the task model processes the data to be processed and outputs a data processing result;

[0027] The data to be processed includes at least one of text data, image data, and voice data, and the task model is obtained by processing using the model optimization method described in any one of the first aspects of the embodiments of the present application.

[0028] Optionally, the method further includes:

[0029] Inputting the data to be processed and the data processing result into an evaluation model for evaluating the processing result of the task model to obtain an evaluation result of the data processing result;

[0030] The evaluation result of the data processing result is input into the task model, so that the task model generates an optimized data processing result corresponding to the data to be processed based on the evaluation result of the data processing result.

[0031] Optionally, the training process of the evaluation model includes:

[0032] Obtaining historical data pairs and evaluation result labels corresponding to each historical data pair, wherein the historical data pairs include historical data and data processing results obtained by the task model processing the historical data; and performing evaluation result generation training on the evaluation model based on the historical data pairs and the evaluation result labels corresponding to each historical data pair;

[0033] and / or,

[0034] Inputting first historical task data and a first data processing result corresponding to the first historical task data into the evaluation model to obtain a first historical evaluation result; inputting the first historical evaluation result into the task model so that the task model generates a second data processing result corresponding to the first historical task data based on the first historical evaluation result; determining a score for the first historical evaluation result based on the first data processing result and the second data processing result; and optimizing parameters of the evaluation model with the goal of improving the score.

[0035] According to a third aspect of an embodiment of the present application, a model optimization device is provided, including:

[0036] A first processing unit is used to input the first task data and the i-th processing result into an evaluation model for evaluating the processing result of the task model, so as to obtain an i-th evaluation result corresponding to the i-th processing result, when an i-th processing result is obtained by the task model processing the first task data; the first task data includes at least one of text data, image data, and voice data, and the task model is used to perform at least one of a text processing task, an image processing task, and a voice processing task;

[0037] A second processing unit is used to input the i-th evaluation result into the task model when it is determined based on the i-th evaluation result that the i-th processing result needs to be optimized, so that the task model processes the first task data based on the i-th evaluation result to obtain the (i+1)th processing result; wherein i is a positive integer.

[0038] According to a fourth aspect of an embodiment of the present application, there is provided a data processing device, including:

[0039] A data processing unit, used for inputting the data to be processed into a pre-trained task model, so that the task model processes the data to be processed and outputs a data processing result;

[0040] The data to be processed includes at least one of text data, image data, and voice data, and the task model is obtained by processing using the model optimization method described in any one of the first aspects of the embodiments of the present application.

[0041] According to a fifth aspect of the embodiments of the present application, there is provided an electronic device, including:

[0042] Memory and processor;

[0043] The memory is connected to the processor and is used to store programs;

[0044] The processor is used to implement the method as described in any one of the first aspect and / or second aspect of the embodiments of the present application by running the program in the memory.

[0045] According to the sixth aspect of the embodiments of the present application, a computer program product is provided, comprising computer program instructions, which, when executed by a processor, enable the processor to execute a method as described in any one of the first aspect and / or second aspect of the embodiments of the present application.

[0046] The model optimization method proposed in this application evaluates the data processing results of the task model through the evaluation model during the data processing of the task model, and then optimizes the data processing results based on the evaluation results of the evaluation model to regenerate new data processing results. By cyclically executing the above process, the output of the task model can be continuously optimized, the quality of the model output can be improved, and the task model can gradually learn the working ability that meets the user's needs. In addition, the above process realizes the automatic optimization and adjustment of the task model, and its model optimization efficiency is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0048] Figure 1 A flow chart of a model optimization method provided in an embodiment of the present application.

[0049] Figure 2 A flowchart of another model optimization method provided in an embodiment of the present application.

[0050] Figure 3 A schematic diagram of the processing flow of a training process of an evaluation model provided in an embodiment of the present application.

[0051] Figure 4 A schematic diagram of the processing flow of another evaluation model training process provided in an embodiment of the present application.

[0052] Figure 5 A flowchart of a data processing method provided in an embodiment of the present application.

[0053] Figure 6 A flowchart of another data processing method provided in an embodiment of the present application.

[0054] Figure 7 A schematic diagram of the structure of a model optimization device provided in an embodiment of the present application.

[0055] Figure 8 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application.

[0056] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The technical solution provided in the embodiments of the present application can be exemplarily applied to hardware devices such as processors, electronic devices, servers (including cloud servers), or packaged into software programs to be run. When the hardware device executes the processing of the technical solution of the embodiments of the present application, or the above-mentioned software program is run, the automatic splitting of the target task and the automatic calling of the application program interface required for the task can be achieved to complete the purpose of the target task. The embodiments of the present application only exemplarily introduce the specific processing of the technical solution of the present application, and do not limit the specific implementation form of the technical solution of the present application. Any technical implementation form that can execute the processing of the technical solution of the present application can be adopted by the embodiments of the present application.

[0058] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0059] Before introducing this application solution, the relevant technologies are first introduced:

[0060] Large language models have shown excellent performance in a variety of data processing tasks due to their excellent understanding, data processing and learning capabilities. However, large language models do not have the ability to meet user needs in all scenarios, and in some cases there are output errors.

[0061] In order to make the large language model have the working ability that meets user needs, the traditional solution is to fine-tune the large language model. However, some large language models are closed or have limited access, and it is impossible to directly fine-tune the model parameters.

[0062] In view of this, the embodiments of the present application are committed to providing a model optimization method, data processing method, device, equipment and program product, which can enable the task model to optimize the data processing results based on the evaluation results of the evaluation model, and are described in detail one by one in the following embodiments.

[0063] Exemplary Methods

[0064] Figure 1 A flow chart of a model optimization method provided in one embodiment of the present application. Figure 1 As shown, the model optimization method provided in this embodiment includes steps S101-S102:

[0065] S101. When an i-th processing result is obtained by obtaining the task model from processing first task data, the first task data and the i-th processing result are input into an evaluation model for evaluating the processing result of the task model to obtain an i-th evaluation result corresponding to the i-th processing result; the first task data includes at least one of text data, image data, and voice data, and the task model is used to perform at least one of a text processing task, an image processing task, and a voice processing task.

[0066] The task model can be understood as a large language model or a multimodal large language model for performing a specific task, which can receive and process multiple types of data, including but not limited to text data, image data, voice data, etc. Optionally, the task model can also process video data. The task model integrates complex algorithms and a large number of parameter sets, which can process input data and generate corresponding processing results based on the input data.

[0067] Depending on the type of input data, the task model can perform different types of tasks. For example, when processing text data, the task model can perform various types of text processing tasks such as text generation, text summary extraction, text translation, and text question and answer; when processing image data, the task model can perform various types of image processing tasks such as image description generation and image classification; when processing speech data, the task model can perform various types of speech processing tasks such as speech recognition and speech synthesis. Optionally, the task model has the ability to process multiple data types at the same time. For example, when performing a visual question and answer task, the model needs to receive and process image data and related text questions at the same time to generate accurate answers.

[0068] The first task data can be understood as the original data input into the task model when the task model performs a specific task. The first task data is in various forms and can be a combination of one or more types of data such as text data, image data, voice data, etc. Optionally, the first task data can also be video data. Optionally, the first task data includes a task instruction that instructs the task model to perform a specific task.

[0069] The i-th processing result can be understood as the corresponding processing result output by the task model after processing the first task data for the i-th time, which can be in various forms such as text, image, voice, etc., depending on the execution task of the task model.

[0070] The evaluation model is designed to evaluate the processing results of the task model based on the input data and processing results of the task model. Specifically, the evaluation model will conduct an in-depth analysis of the input data of the task model and the processing results of the task model, and comprehensively evaluate the processing results of the task model based on at least one evaluation factor (such as the accuracy, completeness, consistency, relevance, etc. of the processing results). In this process, the evaluation model may use various algorithms and technologies (such as machine learning, deep learning, natural language processing, etc.) to achieve accurate evaluation of the processing results of the task model.

[0071] The output of the evaluation model is the evaluation result of the processing result of the task model, which can be a quantitative indicator, score, grade or natural language description, or a detailed evaluation report or suggestion to provide more comprehensive feedback. Optionally, the evaluation result includes feedback on the processing quality of the task model processing result and information such as areas for improvement.

[0072] The evaluation model may be a variety of types of models, for example, a model specifically designed for a certain type of task, or a more powerful large language model or a multimodal large language model.

[0073] Optionally, when the evaluation model is a large language model or a multimodal large language model, the input of the evaluation model, in addition to the input data and output results of the task model, also includes task instructions that instruct the evaluation model to perform evaluation tasks, which are used to guide the evaluation model how to effectively evaluate the processing results of the task model.

[0074] Optionally, the task model and / or the evaluation model may be any type of large language model or multimodal large language model in the prior art.

[0075] Specifically, after the first task data is input into the task model, the task model processes the input first task data according to its internal algorithm and parameters, and outputs the corresponding i-th processing result. Subsequently, the i-th processing result and the first task data are input into the evaluation model together, and the evaluation model conducts an in-depth analysis on them, comprehensively considers at least one evaluation factor (such as the accuracy, completeness, consistency, relevance, etc. of the processing result), evaluates the i-th processing result based on the first task data and the i-th processing result, and then outputs the evaluation result of the i-th processing result, i.e., the i-th evaluation result, to provide processing quality feedback on the i-th processing result of the task model.

[0076] S102. When it is determined based on the i-th evaluation result that the i-th processing result needs to be optimized, the i-th evaluation result is input into the task model so that the task model processes the first task data based on the i-th evaluation result to obtain the (i+1)th processing result; wherein i is a positive integer.

[0077] Before executing step S102 , it is necessary to determine whether the i-th processing result needs to be optimized based on the i-th evaluation result.

[0078] According to the above content, the i-th evaluation result output by the evaluation model includes the evaluation of the i-th processing result, which is the processing quality feedback of the i-th processing result of the task model. By analyzing the i-th evaluation result, it can be determined whether the i-th processing result needs to be optimized.

[0079] Specifically, a judgment standard is preset, and the i-th evaluation result is compared with the judgment standard, and whether the i-th processing result needs to be optimized can be determined according to the comparison result.

[0080] Optionally, the judgment criterion may be a preset score threshold. When the i-th evaluation result is presented in the form of a single score, or a specific score can be obtained by comprehensively analyzing the i-th evaluation result, the score can be regarded as the quality score of the i-th processing result of the task model, and the score is compared with the preset score threshold. When the score is greater than or equal to the preset score threshold, it means that the processing quality of the i-th processing result of the task model has reached the preset quality standard and meets the preset requirements, and there is no need to optimize the i-th processing result; conversely, when the score is less than the preset score threshold, it means that the processing quality of the i-th processing result of the task model has not met the preset requirements, and the i-th processing result needs to be optimized to meet the preset requirements.

[0081] The score obtained by comprehensive analysis of the i-th evaluation result may be a weighted synthesis of multiple indicators to obtain an overall score, or may be converted into a corresponding score or rating through in-depth analysis of the evaluation or suggestion in a specific natural language form (such as semantic analysis and sentiment tendency judgment).

[0082] Optionally, the judgment criteria may include a set of indicator thresholds. For the i-th evaluation result containing multiple quantitative indicators, a threshold may be set for each evaluation indicator. Each indicator in the i-th evaluation result is compared with the corresponding threshold one by one, and a comprehensive judgment is made after all evaluation indicators are compared. If multiple indicators do not reach the threshold, or a key indicator is far below the threshold, it can be considered that the task model has obvious deficiencies in processing the corresponding data, and the i-th processing result needs to be optimized.

[0083] Optionally, if the evaluation result is a natural language description, natural language processing technology is needed to analyze the sentiment tendency and key information therein to determine whether optimization is needed.

[0084] Furthermore, when it is determined based on the i-th evaluation result that the i-th processing result needs to be optimized, the i-th evaluation result is fed back to the task model, and the task model analyzes the i-th evaluation result, optimizes its processing result based on the i-th evaluation result, processes the original first task data again, and generates a new i+1-th processing result. The i+1-th processing result is obtained after considering the feedback of the i-th evaluation result, and is therefore more optimized and accurate than the i-th processing result, such as improved semantic accuracy in text processing, more accurate edge detection in image processing, and improved recognition rate in speech processing. This optimization process is continuous, and each iteration will lead to further optimization of the processing results. In this way, the task model can continuously generate more optimized processing results, thereby better completing various processing tasks.

[0085] As an optional implementation method, the task model optimizes the processing results based on the i-th evaluation results, including: based on the understanding of the i-th evaluation results, starting the automatic learning and optimization process, and actively adjusting its internal parameters or processing strategies to generate a more optimized i+1-th processing result when processing the first task data again.

[0086] Optionally, when the task model receives the i-th evaluation result as input, it will start an automatic learning and optimization process to learn and improve the model performance from the content data of the i-th evaluation result. This process can be implemented based on relevant technologies such as machine learning or deep learning algorithms.

[0087] Specifically, the task model will analyze the content of the i-th evaluation result, determine its own errors or deficiencies in the i-th processing, and then actively adjust its internal parameter settings (such as the weights and biases of the neural network, etc.) and / or processing strategies to correct its errors or deficiencies in the i-th processing, thereby improving the accuracy of processing the first task data. For example, the weights and biases of the neural network are adjusted through the back-propagation algorithm, or the structure or processing strategy of the model is improved through a more complex optimization algorithm.

[0088] After parameter optimization, the task model processes the first task data again to generate a new processing result, i.e., the i+1th processing result. The i+1th processing result is obtained by the task model after considering the feedback of the i-th evaluation result and adjusting the model parameters. Therefore, the i+1th processing result is more optimized than the i-th processing result.

[0089] By continuously feeding the evaluation results back to the task model and performing multiple iterations and optimizations, the parameters or processing strategies of the task model will gradually be adjusted to a state that can produce the best processing results, so that the task model can perform higher performance and accuracy in subsequent processing of similar task data, can process input data more accurately, and generate processing results that are closer to actual expectations. The core of this optimization method is to directly improve at least one of the model's internal parameters, structure, and processing strategies, rather than just adjusting the results at the output level. By making adjustments in-depth within the model, more fundamental performance improvements can be achieved. In addition, this optimization method can achieve automatic optimization and adjustment of task model parameters, and its model optimization efficiency is higher and its scope of application is wider.

[0090] In some application scenarios, frequent adjustment of model parameters may lead to unstable model performance and even introduce new errors.

[0091] As an optional implementation method, the task model optimizes the processing results based on the i-th evaluation result, including: actively adjusting its output processing process based on the understanding of the i-th evaluation result to generate a more optimized i+1-th processing result when processing the first task data again.

[0092] Optionally, adjusting the output processing process includes: adjusting the post-processing steps of the model output, such as highlighting key information by enhancing image details, correcting grammatical errors in text to improve the accuracy of expression, or improving the clarity of speech signals to ensure the complete transmission of information. Optionally, adjusting the output processing process includes: flexibly selecting different decoding strategies according to different task requirements, such as using greedy search to pursue local optimal solutions, or using beam search to explore more possible output sequences, or increasing the diversity of outputs through temperature sampling. These adjustments are intended to refine the processing results, thereby optimizing the overall performance of the model and further improving the quality of the output processing results.

[0093] This optimization method can significantly improve the professionalism and accuracy of the model output results even if the internal parameters of the model have not changed. This provides users with better processing results. This implementation method pays more attention to the quality of the results ultimately received by users and is committed to improving user experience through iterative optimization. It is especially suitable for scenarios that have strict requirements on output results but do not want or need to adjust the parameters of the task model itself. It can maintain the stability of the task model while continuously improving its performance and performance in practical applications.

[0094] As an optional implementation method, the task model optimizes the processing result based on the i-th evaluation result, including: generating a more optimized (i+1)-th processing result according to the i-th evaluation result, the first task data and the i-th processing result.

[0095] Specifically, when the task model performs the processing task on the first task data for the i+1th time, the first task data, the processing result of the i-th processing (i-th processing result), and the evaluation of the processing result (i-th evaluation result) can provide additional context information for the task model to perform the current task, helping the model to better understand and process the current task for the first task data. This means that when the task model processes the first task data, it will take into account the previous processing results and evaluations, so as to make more informed decisions and generate better processing results (i+1th processing results).

[0096] Optionally, the first task data, the i-th processing result and the i-th evaluation result are input into the task model together.

[0097] Optionally, the task model may have acquired the first task data and the i-th processing result from the previous processing process, so there is no need to input them again. In this case, only the i-th evaluation result needs to be input into the model.

[0098] In this implementation, the task model uses the previous input data (first task data), processing results (i-th processing results) and evaluations (i-th evaluation results) as context information to assist the task model in better understanding and processing the current task (i.e., processing the first task data). In this way, the task model can optimize its inference process without changing its basic parameters, structure or result post-processing flow, thereby generating better processing results.

[0099] Although there are different implementations of the above processing result optimization, they can all achieve the core goal of step S102, that is, optimizing the processing result of the task model according to the i-th evaluation result. The above three implementations can be used alone or in combination, depending on the application scenario, model type and optimization requirements.

[0100] The model optimization method proposed in this application evaluates the data processing results of the task model through the evaluation model during the data processing of the task model, and then optimizes the data processing results based on the evaluation results of the evaluation model to regenerate new data processing results. By cyclically executing the above processing, the output of the task model can be continuously optimized, the quality of the model output can be improved, and the task model can gradually learn the working ability that meets user needs. In addition, the above process can realize automatic optimization and adjustment of the task model parameters or post-processing processes, and its model optimization efficiency is higher.

[0101] Figure 2 A flow chart of another model optimization method provided in an embodiment of the present application. Figure 2As shown, as an optional implementation, the model optimization method provided in the embodiment of the present application further includes steps S201-S202:

[0102] S201. When it is determined based on the kth evaluation result output by the evaluation model that the kth processing result output by the task model meets the preset processing result requirements, use the first task data and the kth processing result to form a training data pair; k is a positive integer.

[0103] As an optional implementation, processing result requirements are preset, and these requirements may be thresholds regarding accuracy, completeness, consistency or other quality indicators.

[0104] Optionally, the processing result requirement is the same as the judgment criterion for determining whether the i-th processing result needs to be optimized based on the i-th evaluation result in step S102 of the aforementioned embodiment.

[0105] Each time the evaluation model evaluates the processing results output by the task model, after generating the evaluation results, the generated evaluation results are compared with the preset processing result requirements. If the comparison result indicates that the processing results output by the task model in this processing meet the preset processing result requirements, then the processing result and its corresponding input data (i.e., the first task data) are combined into a training data pair.

[0106] As an optional implementation method, the evaluation model evaluates the processing result output by the task model, and the evaluation result includes a judgment indicating whether the processing result output by the task model meets the preset processing result requirements. If the evaluation result shows that the processing result output by the task model in this processing meets the preset processing result requirements, then the processing result and its corresponding input data (i.e., the first task data) are combined into a training data pair.

[0107] The training data pair includes input data and its corresponding output processing result that has been confirmed to meet the preset result requirements, representing an instance in which the task model can produce processing results that meet the requirements under specific input. The training data pair can be regarded as a high-quality training data pair.

[0108] Optionally, when the preset processing result requirement is set high enough, a training data pair with sufficiently high quality will be obtained. For example, when the comment model believes that the task model response is accurate, the training data pair consisting of the corresponding first task data and the processing result of the task model can be regarded as the training data pair with the highest quality.

[0109] Furthermore, when it is determined based on the kth evaluation result output by the evaluation model that the kth processing result output by the task model does not meet the preset processing result requirements, the kth evaluation result is input into the task model so that the task model processes the first task data based on the kth evaluation result to obtain the k+1th processing result, and again judges whether the k+1th processing result output by the task model meets the preset processing result requirements based on the kth evaluation result output by the evaluation model, and repeats the above steps until it is determined that the processing result output by the task model meets the preset processing result requirements.

[0110] S202: Utilize the training data pair to perform optimization training on the task model.

[0111] The training data pairs {first task data, kth processing result} obtained in step S201 are all high-quality training data pairs that meet the preset processing result requirements. Using these training data pairs to optimize the task model can enhance the ability of the task model to produce qualified results when processing similar inputs.

[0112] As an optional implementation method, the task model is optimized and trained using high-quality training data pairs, including: using the first task data in the training data pair {first task data, kth processing result} as a training sample and the kth processing result as a sample label to perform supervised training on the task model.

[0113] Specifically, the first task data is input into the task model as a training sample, and the task model performs the corresponding specific task, processes the first task data, and outputs the processing result of the first task data. According to the processing result output by the task model and the kth processing result as the sample label, the loss function value is determined, and according to the loss function value, the parameters of the task model are optimized by a back propagation algorithm or other methods to minimize the difference between the processing result output by the task model and the sample label.

[0114] This implementation method can clearly guide the task model to learn the mapping relationship from input to expected output through supervised training, so that the task model can more accurately learn how to process similar inputs to produce outputs that meet the requirements.

[0115] As an optional implementation method, high-quality training data pairs are used to optimize the training of the task model, including: taking the training data pair {first task data, kth processing result} as a whole as a training sample, directly inputting it into the task model, and performing unsupervised training on the task model.

[0116] Specifically, the task model's powerful self-learning and representation learning capabilities are used to explore the rules and patterns in the input training data. By mining the internal correlation, distribution patterns or clustering structure of the training data, a series of training and learning are carried out to gradually improve its own processing capabilities for similar inputs and generate processing results that better meet the preset requirements.

[0117] In practical applications, you can choose a suitable optimization training method or combine the two optimization training methods according to the specific application scenarios and requirements to achieve better results.

[0118] In this embodiment, through step S201 and step S202, the system can effectively screen out high-quality processing results and combine them with the original input data into training data pairs, providing strong support for subsequent task model optimization training. This method can ensure that the task model learns the correct and high-quality data mapping relationship during the training process, thereby improving the performance and accuracy of the model.

[0119] As an optional implementation method, the evaluation model is trained to generate evaluation results based on historical data pairs and evaluation result labels corresponding to each historical data pair, wherein the historical data pairs include historical task data and data processing results obtained by the task model processing the historical task data.

[0120] Specifically, Figure 3 As shown, the training process of the evaluation model includes steps S301-S302:

[0121] S301. Obtain historical data pairs and evaluation result labels corresponding to each historical data pair, wherein the historical data pairs include historical data and data processing results obtained by processing the historical data by the task model.

[0122] Each historical data pair consists of two parts. One part is the historical data, that is, the first task data input into the task model at a certain point in the past; the other part is the data processing result output by the task model after completing the processing of the historical data.

[0123] The historical data pairs reflect various input and output situations encountered by the model in actual applications.

[0124] For each historical data pair, an evaluation result label is set. The evaluation result label can be manually annotated for each historical data pair by experts or personnel with rich domain knowledge according to the processing results of the task model, or can be generated by other automated evaluation methods.

[0125] The evaluation result label provides an objective standard for evaluating the processing effect of the task model and provides important guidance information for the training of the evaluation model.

[0126] The evaluation result label can be a binary label (such as "good" or "bad"), a more fine-grained label (such as specific values ​​such as accuracy and recall), or a multi-classification label, depending on the complexity and requirements of the evaluation model.

[0127] After obtaining these data, some preprocessing work is usually required, such as data cleaning and standardization, to ensure the quality and consistency of the data and prepare for the subsequent evaluation model training.

[0128] S302: Based on the historical data pairs and the evaluation result labels corresponding to each historical data pair, the evaluation model is trained to generate evaluation results.

[0129] Specifically, the historical data pair is input into the evaluation model, and the evaluation model will evaluate the input historical data pair according to its current parameters and algorithms to generate a predicted evaluation result.

[0130] The evaluation result generated by the evaluation model is compared with the evaluation result label corresponding to the historical data pair, and the loss function value is determined according to the difference between the two. The optimization algorithm (such as gradient descent method, etc.) is used to optimize the parameters of the evaluation model according to the loss function value to reduce the difference between the evaluation result predicted by the evaluation model and the actual evaluation result label.

[0131] Repeat the above process and continuously train the model through multiple epochs until the evaluation result of the model reaches a certain accuracy or the loss function converges to a smaller value.

[0132] Through the above training process, the evaluation model can learn how to accurately generate evaluation results based on the input data pair (including the first task data and the processing result obtained by the task model processing the first task data). When a new task model processing result appears, the evaluation model can quickly and accurately give a corresponding evaluation, thereby helping the system understand the processing effect of the task model.

[0133] As an optional implementation, the training process of the evaluation model further includes steps S401-S404:

[0134] S401: Input first historical task data and a first data processing result corresponding to the first historical task data into the evaluation model to obtain a first historical evaluation result.

[0135] The first historical task data may be understood as first task data that was input into the task model at a certain moment in the past and has been processed by the task model.

[0136] The first data processing result can be understood as the processing result output by the task model after processing the first historical task data.

[0137] The first historical task data and the first data processing result of the first historical task data are input into the evaluation model. The evaluation model will perform evaluation processing based on the input first historical task data and its corresponding first data processing result, and output a first historical evaluation result for the first data processing result.

[0138] S402: Input the first historical evaluation result into the task model, so that the task model generates a second data processing result corresponding to the first historical task data based on the first historical evaluation result.

[0139] The first historical evaluation result includes an evaluation of the first data processing result. The first historical evaluation result is input into the task model, and the task model processes the first historical task data again based on the first historical evaluation result to obtain a second data processing result of the first historical task data.

[0140] As for the specific implementation of the task model processing the first historical task data again based on the first historical evaluation result to obtain the second data processing result of the first historical task data, reference may be made to the relevant content of step S102 in the aforementioned embodiment, which will not be elaborated here.

[0141] S403: Determine a score for the first historical evaluation result based on the first data processing result and the second data processing result.

[0142] According to the above content, the first data processing result is the result obtained by the task model processing the first historical task data without feedback from the evaluation model. The second data processing result is the result obtained by the task model reprocessing the same first historical task data after receiving feedback from the evaluation model (i.e., the first historical evaluation result).

[0143] After determining the first data processing result and the second data processing result, the two sets of data processing results are compared. The comparison result reflects whether the processing result of the task model has been improved and the degree of improvement after receiving the feedback from the evaluation model (i.e., the first historical evaluation result), and the improvement is based on the first historical evaluation result.

[0144] By comparing the differences or improvements between the processing results of the task model before and after it receives the feedback from the evaluation model (i.e., the first historical evaluation result), it is possible to quantitatively evaluate the feedback effect provided by the evaluation model and present the feedback effect in the form of a score, that is, the score is a quantitative evaluation of the feedback effect provided by the evaluation model.

[0145] Optionally, determining the score for the first historical evaluation result based on the first data processing result and the second data processing result can be achieved by calculating the similarity, accuracy or other relevant metrics between the two, and determining the score for the first historical evaluation result based on the calculation result.

[0146] Optionally, the score of the first historical evaluation result is determined based on the first data processing result and the second data processing result, and can be determined by calculating the difference between the first score of the first data processing result and the second score of the second data processing result. This will be explained in subsequent embodiments and will not be described in detail here.

[0147] S404: Optimize parameters of the evaluation model with the goal of improving the score.

[0148] In step S403, the score of the first historical evaluation result has been determined based on the first data processing result and the second data processing result. This score is actually a quantitative evaluation of the feedback effect provided by the evaluation model, reflecting the contribution of the evaluation model to the improvement of the task model performance.

[0149] In order to improve the score of the first historical evaluation result output by the evaluation model, it is necessary to further optimize the parameters of the evaluation model, which usually involves the use of an optimization algorithm, such as gradient descent, genetic algorithm or other optimization techniques in machine learning, which is not limited in this application.

[0150] As an optional implementation method, a reinforcement learning method is used to optimize the parameters of the evaluation model. Various types of reinforcement learning methods in the prior art can be used, and this application does not limit this.

[0151] Optionally, a reinforcement learning method based on proximal policy optimization (PPO) is used to optimize the parameters of the evaluation model so that it is optimized in the direction of maximizing the reward and generating higher quality reviews. In this optimization process, the loss function is calculated according to the following formula:

[0152]

[0153] Among them, L PPO represents the loss function value, π actor and π ref Both represent the likelihood probability of the evaluation model output, the difference is πactor represents the evaluation model currently being optimized, π ref represents the previously optimized evaluation model, a t Indicates the word to be predicted at present, s t Indicates the predicted word a t A represents the score of the first historical evaluation result, and λ is a training parameter used to adjust the proportion of SFT loss.

[0154] Optimizing the parameters of the evaluation model is usually an iterative process. In each iteration, the parameters of the evaluation model are updated according to the current score and adjustment strategy of the first historical rating result, and then steps S401-S403 are rerun to evaluate the performance of the evaluation model under the new parameter settings and observe whether the new score is improved. The optimization process will continue until a certain stop condition is met, such as the score reaches a preset threshold, the improvement of the optimization process is less than a threshold, or the preset maximum number of iterations is reached.

[0155] Through the parameter optimization process of step S404, a more accurate and effective evaluation model can be obtained, thereby better assisting the task model to improve its performance.

[0156] As an optional implementation, step S403 "determining a score for the first historical evaluation result based on the first data processing result and the second data processing result" includes steps A1-A2:

[0157] A1. Score the first data processing result to obtain a first score, and score the second data processing result to obtain a second score.

[0158] Specifically, the first data processing result is scored according to a preset scoring standard, algorithm or model to obtain a first score. The first score reflects the processing effect of the task model on the first historical task data when no evaluation model feedback (ie, the first historical evaluation result) is received.

[0159] The second data processing result is scored using the same scoring criteria, algorithm or model as that used to score the first data processing result, to obtain a second score. The second score reflects the processing effect of the task model on the first historical task data after receiving the evaluation model feedback (i.e., the first historical evaluation result).

[0160] As an optional implementation, the first score and the second score are implemented based on a model-based processing concept.

[0161] As an optional implementation method, step A1 includes: inputting the first historical task data and the first data processing result into a pre-trained reward model to obtain a first score corresponding to the first data processing result; and inputting the first historical task data and the second data processing result into the reward model to obtain a second score corresponding to the second data processing result.

[0162] Specifically, a reward model is pre-trained, which is used to evaluate the effect of the task model in processing data and give a quantitative score.

[0163] Optionally, the reward model is obtained by scoring training based on the first training data and the scoring label corresponding to the first training data. The first training data includes the input data of the task model and the data processing result of the task model on the input data. The scoring training is to enable the reward model to predict and score the data processing result based on the input data of the task model and the data processing result of the task model on the input data, and to enable the predicted score determined by the reward model and the loss function value determined by the scoring label to meet the preset requirements.

[0164] Optionally, the reward model can be implemented based on a variety of deep learning network structures, such as CNN network structure, RNN network structure, CRNN network structure, etc., which is not limited in this application.

[0165] This reward model-based scoring method provides a more objective and quantitative evaluation method, which helps to understand the performance of the evaluation model more accurately and provides strong support for subsequent optimization of the evaluation model.

[0166] A2. Calculate the difference between the first score and the second score as the score for the first historical evaluation result.

[0167] The difference between the first score and the second score reflects the influence of the evaluation model feedback on the processing effect of the task model.

[0168] For example, if the difference is positive, it means that the feedback of the evaluation model has a positive improvement effect on the task model; if the difference is negative, it means that the feedback may have led to a decline in performance; if the difference is zero, the feedback has no significant effect.

[0169] Through steps A1 and A2, the impact of the evaluation model feedback on the task model performance can be quantitatively evaluated, thus providing a clear direction and basis for subsequent optimization of the evaluation model. This scoring difference method is more intuitive and easy to understand, and can directly reflect the effectiveness of the evaluation model.

[0170] Figure 5A flow chart of a data processing method provided in an embodiment of the present application. The data processing method provided in this embodiment can be applied to a variety of application scenarios such as text processing (such as sentiment analysis, text classification, machine translation), image processing (such as image recognition, target detection, image generation), speech processing (such as speech recognition, speech synthesis), multimodal data processing (such as video content understanding, multimodal sentiment analysis), automation and robotics (such as autonomous navigation), and recommendation systems. The specific application scenario is determined based on the specific task that the task model is trained to perform.

[0171] like Figure 5 As shown, the data processing method provided in the embodiment of the present application includes step S501:

[0172] S501. Input the data to be processed into a pre-trained task model so that the task model processes the data to be processed and outputs a data processing result; wherein the data to be processed includes at least one of text data, image data, and voice data, and the task model is obtained by processing using the model optimization method provided by any of the aforementioned embodiments.

[0173] Specifically, after the data to be processed is input into the pre-trained task model, the task model will use its learned parameters and algorithms to process the data to be processed and output the data processing results.

[0174] The data to be processed is in various forms, and can be a combination of one or more types of data such as text data, image data, voice data, etc. Optionally, the data to be processed can also be video data.

[0175] Optionally, the data to be processed also includes task instructions that instruct the task model to perform a specific task.

[0176] The data processing result can be understood as the corresponding processing result output by the task model after processing the data to be processed, which can be in various forms such as text, image, voice, etc., depending on the type of task model and the designed task objectives.

[0177] In this embodiment, since the task model is optimized by the model optimization method provided in the above-mentioned embodiment, the task model undergoes a series of optimization processes, which may involve one or more aspects such as adjustment of model parameters, improvement of model structure, optimization of processing strategy, improvement of post-processing flow, and improvement of decoding strategy. These optimization measures work together to improve the quality of the task processing results output by the task model, so that the task model is more accurate and efficient when performing the designed tasks. Therefore, the pre-trained task model can better perform its designed tasks and output more optimized and higher quality data processing results based on the data to be processed. Furthermore, since the data processing results output by the task model are more optimized and of higher quality, users will have a better experience when using these data processing results, thereby improving the user's satisfaction and trust in the task model.

[0178] Figure 6 A flowchart of another data processing method provided in an embodiment of the present application. As an optional implementation method, Figure 6 As shown, the data processing method provided in this embodiment includes steps S601-S603:

[0179] S601. Input the data to be processed into a pre-trained task model so that the task model processes the data to be processed and outputs a data processing result, wherein the data to be processed includes at least one of text data, image data, and voice data, and the task model is obtained by processing using the model optimization method provided by any of the aforementioned embodiments.

[0180] The specific content of step S601 corresponds to the content of step S501 in the aforementioned embodiment. Please refer to the description of step S501 in the aforementioned embodiment, and no further details will be given here.

[0181] S602: Input the data to be processed and the data processing result into an evaluation model for evaluating the processing result of the task model to obtain an evaluation result of the data processing result.

[0182] For specific contents of the evaluation model and the evaluation result of the data processing result, reference may be made to the relevant description of step S101 in the aforementioned embodiment, which will not be elaborated here.

[0183] Specifically, the data to be processed and the data processing results of the task model on the data to be processed are input into the evaluation model. The evaluation model will conduct an in-depth analysis on them, comprehensively consider at least one evaluation factor (such as the accuracy, completeness, consistency, relevance, etc. of the processing results), evaluate the data processing results based on the data to be processed and the data processing results, and then output the evaluation results of the data processing results to provide processing quality feedback on the data processing results of the task model.

[0184] S603: Input the evaluation result of the data processing result into the task model, so that the task model generates an optimized data processing result corresponding to the data to be processed based on the evaluation result of the data processing result.

[0185] For the specific content of step S603, reference may be made to the relevant description of step S102 in the aforementioned embodiment, which will not be elaborated here.

[0186] Specifically, the evaluation result of the data processing result is fed back to the task model, and the task model will analyze the evaluation result, and process the data to be processed again based on the evaluation result to generate a new data processing result. Because the new data processing result is obtained after considering the feedback of the evaluation result, it is more optimized and accurate than the data processing result obtained in the previous processing. For example, the semantic accuracy of text processing is improved, edge detection in image processing is more precise, and the recognition rate in speech processing is improved.

[0187] Optionally, in actual applications, steps S602-S603 may be executed repeatedly to obtain more optimized and higher quality data processing results and better complete various processing tasks. As for the number of executions of the loop, it is determined according to the actual application scenario and requirements, and this application does not limit this.

[0188] As an optional implementation, Figure 6 The training process of the evaluation model in the illustrated embodiment includes:

[0189] Obtaining historical data pairs and evaluation result labels corresponding to each historical data pair, wherein the historical data pairs include historical data and data processing results obtained by the task model processing the historical data; and performing evaluation result generation training on the evaluation model based on the historical data pairs and the evaluation result labels corresponding to each historical data pair;

[0190] and / or,

[0191] Inputting first historical task data and a first data processing result corresponding to the first historical task data into the evaluation model to obtain a first historical evaluation result; inputting the first historical evaluation result into the task model so that the task model generates a second data processing result corresponding to the first historical task data based on the first historical evaluation result; determining a score for the first historical evaluation result based on the first data processing result and the second data processing result; and optimizing parameters of the evaluation model with the goal of improving the score.

[0192] The above training process is Figure 3 The illustrated embodiments and Figure 4For the contents of the illustrated embodiments, please refer to Figure 3 The illustrated embodiments and Figure 4 The description of the illustrated embodiment will not be repeated here.

[0193] Exemplary Devices

[0194] Corresponding to the above-mentioned model optimization method, an embodiment of the present application also provides a model optimization device. Figure 7 is a schematic diagram of the structure of a model optimization device provided in an embodiment of the present application, such as Figure 7 As shown, the model optimization device provided in the embodiment of the present application includes:

[0195] The first processing unit 701 is used for, when an i-th processing result obtained by processing first task data by a task model is obtained, inputting the first task data and the i-th processing result into an evaluation model for evaluating the processing result of the task model, to obtain an i-th evaluation result corresponding to the i-th processing result; the first task data includes at least one of text data, image data, and voice data, and the task model is used to perform at least one of a text processing task, an image processing task, and a voice processing task;

[0196] The second processing unit 702 is used to input the i-th evaluation result into the task model when it is determined based on the i-th evaluation result that the i-th processing result needs to be optimized, so that the task model processes the first task data based on the i-th evaluation result to obtain the (i+1)th processing result; wherein i is a positive integer.

[0197] The model optimization device proposed in the present application evaluates the data processing results of the task model through the evaluation model during the data processing of the task model, and then optimizes the data processing results based on the evaluation results of the evaluation model to regenerate new data processing results. By executing the above process in a loop, the output of the task model can be continuously optimized, the quality of the model output can be improved, and the task model can gradually learn the working ability that meets the user's needs. In addition, the above process realizes the automatic optimization and adjustment of the task model, and its model optimization efficiency is higher.

[0198] Optionally, the device further comprises:

[0199] A third processing unit is used to use the first task data and the kth processing result to form a training data pair when it is determined based on the kth evaluation result output by the evaluation model that the kth processing result output by the task model meets the preset processing result requirements; and use the training data pair to optimize the training of the task model, where k is a positive integer.

[0200] Optionally, the training process of the evaluation model includes:

[0201] Acquire historical data pairs and evaluation result labels corresponding to each historical data pair, wherein the historical data pairs include historical data and data processing results obtained by the task model processing the historical data;

[0202] Based on the historical data pairs and the evaluation result labels corresponding to each historical data pair, the evaluation model is trained to generate evaluation results.

[0203] Optionally, the training process of the evaluation model further includes:

[0204] Inputting the first historical task data and the first data processing result corresponding to the first historical task data into the evaluation model to obtain a first historical evaluation result;

[0205] inputting the first historical evaluation result into the task model so that the task model generates a second data processing result corresponding to the first historical task data based on the first historical evaluation result;

[0206] Determining a score for the first historical evaluation result based on the first data processing result and the second data processing result;

[0207] With the goal of improving the score, the parameters of the evaluation model are optimized.

[0208] Optionally, determining a score for the first historical evaluation result based on the first data processing result and the second data processing result includes:

[0209] Scoring the first data processing result to obtain a first score, and scoring the second data processing result to obtain a second score;

[0210] A difference between the first score and the second score is calculated as a score for the first historical evaluation result.

[0211] Optionally, scoring the first data processing result to obtain a first score, and scoring the second data processing result to obtain a second score, includes:

[0212] Inputting the first historical task data and the first data processing result into a pre-trained reward model to obtain a first score corresponding to the first data processing result;

[0213] And, the first historical task data and the second data processing result are input into the reward model to obtain a second score corresponding to the second data processing result.

[0214] The model optimization device provided in this embodiment belongs to the same application concept as the model optimization method provided in the above embodiments of this application, and can execute the model optimization method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of executing the model optimization method. For technical details not fully described in this embodiment, please refer to the specific processing content of the model optimization method provided in the above embodiments of this application, and will not be repeated here.

[0215] Corresponding to the above-mentioned data processing method, an embodiment of the present application also provides a data processing device. Figure 8 is a structural diagram of a data processing device provided in an embodiment of the present application, such as Figure 8 As shown, the data processing device provided in the embodiment of the present application includes:

[0216] The data processing unit 801 is used to input the data to be processed into a pre-trained task model so that the task model processes the data to be processed and outputs a data processing result;

[0217] The data to be processed includes at least one of text data, image data, and voice data, and the task model is obtained by processing using the model optimization method provided by any of the aforementioned embodiments.

[0218] In this embodiment, since the task model is optimized by the model optimization method provided in the above-mentioned embodiment, the task model undergoes a series of optimization processes, which may involve one or more aspects such as adjustment of model parameters, improvement of model structure, optimization of processing strategy, improvement of post-processing flow, and improvement of decoding strategy. These optimization measures work together to improve the quality of the task processing results output by the task model, so that the task model is more accurate and efficient when performing the designed tasks. Therefore, the pre-trained task model can better perform its designed tasks and output more optimized and higher quality data processing results based on the data to be processed. Furthermore, since the data processing results output by the task model are more optimized and of higher quality, users will have a better experience when using these data processing results, thereby improving the user's satisfaction and trust in the task model.

[0219] Optionally, the device further comprises:

[0220] a fourth processing unit, configured to input the data to be processed and the data processing result into an evaluation model for evaluating the processing result of the task model, to obtain an evaluation result of the data processing result;

[0221] The fifth processing unit is used to input the evaluation result of the data processing result into the task model, so that the task model generates an optimized data processing result corresponding to the data to be processed based on the evaluation result of the data processing result.

[0222] Optionally, the training process of the evaluation model includes:

[0223] Obtaining historical data pairs and evaluation result labels corresponding to each historical data pair, wherein the historical data pairs include historical data and data processing results obtained by the task model processing the historical data; and performing evaluation result generation training on the evaluation model based on the historical data pairs and the evaluation result labels corresponding to each historical data pair;

[0224] and / or,

[0225] Inputting first historical task data and a first data processing result corresponding to the first historical task data into the evaluation model to obtain a first historical evaluation result; inputting the first historical evaluation result into the task model so that the task model generates a second data processing result corresponding to the first historical task data based on the first historical evaluation result; determining a score for the first historical evaluation result based on the first data processing result and the second data processing result; and optimizing parameters of the evaluation model with the goal of improving the score.

[0226] The functions implemented by the above-mentioned units can be implemented by the same or different processors respectively, which is not limited in the embodiments of the present application.

[0227] It should be understood that the units in the above devices can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device, wherein the processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory in the device or a memory outside the device. Alternatively, the units in the device can be implemented in the form of hardware circuits, and the functions of some or all units can be realized by designing the hardware circuits. The hardware circuit can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be implemented by PLD, taking FPGA as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by the configuration file, so as to realize the functions of some or all of the above units. All units of the above devices can be implemented in the form of a processor calling software, or in the form of hardware circuits, or in part by a processor calling software, and the remaining part is implemented in the form of hardware circuits.

[0228] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a GPU, or a DSP; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0229] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0230] In addition, all or part of the units in the above device can be integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a SOC. The SOC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The type of the at least one processor may be different, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0231] Exemplary Electronic Devices

[0232] Another embodiment of the present application also provides an electronic device, see Fig. 9 As shown, the device includes:

[0233] Memory 200 and processor 210;

[0234] The memory 200 is connected to the processor 210 and is used to store programs;

[0235] The processor 210 is used to implement the model optimization method and / or data processing method disclosed in any of the above embodiments by running the program stored in the memory 200.

[0236] Specifically, the electronic device may further include: a bus, a communication interface 220 , an input device 230 and an output device 240 .

[0237] The processor 210, the memory 200, the communication interface 220, the input device 230 and the output device 240 are connected to each other via a bus.

[0238] A bus may include a pathway that transfers information between components of a computer system.

[0239] The processor 210 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the scheme of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0240] The processor 210 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0241] The memory 200 stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes a computer operation instruction. More specifically, the memory 200 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0242] The input device 230 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0243] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0244] The communication interface 220 may include any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0245] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement each step of any model optimization method and / or data processing method provided in the above embodiments of the present application.

[0246] An embodiment of the present application also proposes a chip, which includes a processor and a data interface. The processor reads and runs a program stored in a memory through the data interface to execute the model optimization method and / or data processing method introduced in any of the above embodiments. The specific processing process and its beneficial effects can be found in the introduction to the embodiments of the above-mentioned model optimization method and / or data processing method.

[0247] Exemplary computer program products and storage media

[0248] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the model optimization method and / or data processing method according to various embodiments of the present application described in any of the above embodiments of this specification.

[0249] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0250] In addition, an embodiment of the present application may also be a storage medium on which a computer program is stored, and the computer program is executed by a processor to execute the steps of the model optimization method and / or data processing method according to various embodiments of the present application described in any of the above embodiments of this specification.

[0251] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0252] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0253] The steps in the methods of each embodiment of the present application can be adjusted in order, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0254] The modules and sub-modules in the devices and terminals of the various embodiments of the present application can be combined, divided and deleted according to actual needs.

[0255] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0256] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0257] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0258] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0259] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0260] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0261] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A model optimization method, characterized in that: include: In the case of obtaining an i-th processing result obtained by processing the first task data by the task model, inputting the first task data and the i-th processing result into an evaluation model for evaluating the processing result of the task model, to obtain an i-th evaluation result corresponding to the i-th processing result; the first task data includes at least one of text data, image data, and voice data, and the task model is used to perform at least one of a text processing task, an image processing task, and a voice processing task; When it is determined based on the i-th evaluation result that the i-th processing result needs to be optimized, the i-th evaluation result is input into the task model so that the task model processes the first task data based on the i-th evaluation result to obtain the i+1th processing result; wherein i is a positive integer.

2. The method according to claim 1, characterized in that The method further comprises: In the case where it is determined based on the kth evaluation result output by the evaluation model that the kth processing result output by the task model meets the preset processing result requirements, the first task data and the kth processing result are used to form a training data pair; k is a positive integer; The task model is optimized and trained using the training data pairs.

3. The method according to claim 1 or 2, characterized in that: The training process of the evaluation model includes: Acquire historical data pairs and evaluation result labels corresponding to each historical data pair, wherein the historical data pairs include historical data and data processing results obtained by the task model processing the historical data; Based on the historical data pairs and the evaluation result labels corresponding to each historical data pair, the evaluation model is trained to generate evaluation results.

4. The method according to claim 3, characterized in that The training process of the evaluation model further includes: Inputting the first historical task data and the first data processing result corresponding to the first historical task data into the evaluation model to obtain a first historical evaluation result; inputting the first historical evaluation result into the task model so that the task model generates a second data processing result corresponding to the first historical task data based on the first historical evaluation result; Determining a score for the first historical evaluation result based on the first data processing result and the second data processing result; With the goal of improving the score, the parameters of the evaluation model are optimized.

5. The method according to claim 4, characterized in that Determining a score for the first historical evaluation result based on the first data processing result and the second data processing result includes: Scoring the first data processing result to obtain a first score, and scoring the second data processing result to obtain a second score; A difference between the first score and the second score is calculated as a score for the first historical evaluation result.

6. The method according to claim 5, characterized in that Scoring the first data processing result to obtain a first score, and scoring the second data processing result to obtain a second score, includes: Inputting the first historical task data and the first data processing result into a pre-trained reward model to obtain a first score corresponding to the first data processing result; And, the first historical task data and the second data processing result are input into the reward model to obtain a second score corresponding to the second data processing result.

7. A data processing method, characterized in that: include: Inputting the data to be processed into a pre-trained task model so that the task model processes the data to be processed and outputs a data processing result; The data to be processed includes at least one of text data, image data, and voice data, and the task model is obtained by processing using the model optimization method described in any one of claims 1 to 6.

8. The method according to claim 7, characterized in that The method further comprises: Inputting the data to be processed and the data processing result into an evaluation model for evaluating the processing result of the task model to obtain an evaluation result of the data processing result; The evaluation result of the data processing result is input into the task model, so that the task model generates an optimized data processing result corresponding to the data to be processed based on the evaluation result of the data processing result.

9. The method according to claim 8, characterized in that The training process of the evaluation model includes: Obtaining historical data pairs and evaluation result labels corresponding to each historical data pair, wherein the historical data pairs include historical data and data processing results obtained by the task model processing the historical data; and performing evaluation result generation training on the evaluation model based on the historical data pairs and the evaluation result labels corresponding to each historical data pair; and / or, Input first historical task data and a first data processing result corresponding to the first historical task data into the evaluation model to obtain a first historical evaluation result; input the first historical evaluation result into the task model so that the task model generates a second data processing result corresponding to the first historical task data based on the first historical evaluation result; determine a score for the first historical evaluation result based on the first data processing result and the second data processing result; and optimize parameters of the evaluation model with the goal of improving the score.

10. A model optimization device, characterized in that: include: A first processing unit is used to input the first task data and the i-th processing result into an evaluation model for evaluating the processing result of the task model, so as to obtain an i-th evaluation result corresponding to the i-th processing result, when an i-th processing result is obtained by the task model processing the first task data; the first task data includes at least one of text data, image data, and voice data, and the task model is used to perform at least one of a text processing task, an image processing task, and a voice processing task; A second processing unit is used to input the i-th evaluation result into the task model when it is determined based on the i-th evaluation result that the i-th processing result needs to be optimized, so that the task model processes the first task data based on the i-th evaluation result to obtain the (i+1)th processing result; wherein i is a positive integer.

11. A data processing device, characterized in that: include: A data processing unit, used for inputting the data to be processed into a pre-trained task model, so that the task model processes the data to be processed and outputs a data processing result; The data to be processed includes at least one of text data, image data, and voice data, and the task model is obtained by processing using the model optimization method described in any one of claims 1 to 6.

12. An electronic device, characterized in that: include: Memory and processor; The memory is connected to the processor and is used to store programs; The processor is configured to implement the method according to any one of claims 1 to 9 by running the program in the memory.

13. A computer program product, characterized in that The method comprises computer program instructions which, when executed by a processor, cause the processor to perform the method as claimed in any one of claims 1 to 9.

Images