Data processing method and device and model optimization method and device

Abstract

The embodiment of the invention provides a data processing method and device and a model optimization method and device, and the data processing method comprises the steps: carrying out the sampling in a plurality of data sets, and determining target sample data according to a sampling result; inputting the target sample data into a plurality of language models for processing respectively, and obtaining target prediction data output by the plurality of language models respectively; performing text annotation processing on each piece of target prediction data by using a preset text annotation model to obtain text annotation information corresponding to each piece of target prediction data; based on the target sample data, the target prediction data and the text annotation information, a sample data set is constructed, and the sample data set is used for executing a model training task and a model verification task.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of machine learning technology, and in particular, to data processing methods and devices, and model optimization methods and devices. Background Art

[0002] With the development of computer and Internet technology, large language models have made significant progress in the field of natural language processing. A well-trained large language model can support users in personalized question and answer, thereby meeting the user's usage needs in different scenarios. However, LLMs that rely solely on imitation learning have a significant problem, that is, they may generate content that seems reasonable but is actually inaccurate or unsafe. In the prior art, in order to solve this problem, human feedback information is usually used to optimize the model, such as by collecting a large amount of user evaluation data on the model output, and using a reward model in combination with the evaluation data to optimize the model so that the output it generates is more in line with user needs. However, this optimization method is overly dependent on user feedback content, and the data scale is small and lacks diversity. Therefore, there is an urgent need for an effective solution to solve the above problems. Summary of the invention

[0003] In view of this, an embodiment of this specification provides a data processing method. One or more embodiments of this specification also relate to a data processing device, a model optimization method, a model optimization device, a computing device, a computer-readable storage medium, and a computer program product to solve the technical defects existing in the prior art.

[0004] According to a first aspect of an embodiment of this specification, a data processing method is provided, including:

[0005] Sampling is performed in multiple data sets respectively, and target sample data is determined according to the sampling results;

[0006] Inputting the target sample data into a plurality of language models for processing respectively, and obtaining target prediction data outputted by the plurality of language models respectively;

[0007] Perform text annotation processing on each target prediction data using a preset text annotation model to obtain text annotation information corresponding to each target prediction data;

[0008] A sample data group is constructed based on the target sample data, the target prediction data and the text annotation information, wherein the sample data group is used to perform a model training task and a model verification task.

[0009] Optionally, the sampling is performed in the plurality of data sets respectively, and the target sample data is determined according to the sampling results, including:

[0010] Determine a plurality of data sets, wherein the data types of the sample data contained in each data set are different;

[0011] Sampling is performed in each data set to obtain a plurality of initial sample data, and an initial sample data set is constructed based on the plurality of initial sample data;

[0012] Select target sample data from the initial sample data set.

[0013] Optionally, before the step of inputting the target sample data into a plurality of language models for processing respectively and obtaining the target prediction data outputted by the plurality of language models respectively is executed, the step further includes:

[0014] Selecting multiple language models for the target sample data, and determining prompt information corresponding to each language model;

[0015] The processing of the target sample data by any one of the multiple language models includes:

[0016] The target sample data is updated using prompt information corresponding to the language model, and model input context information is generated according to the update result;

[0017] The model input context information is input into the language model for processing to obtain target prediction data.

[0018] Optionally, determining text annotation information of any one of the target prediction data respectively corresponding to the multiple language models includes:

[0019] Inputting the target sample data and the target prediction data into a preset text annotation model, processing the target prediction data in at least two annotation dimensions using the text annotation model, and determining the prediction scores corresponding to the annotation text and the at least two annotation dimensions respectively according to the processing results;

[0020] Based on the annotation text and the prediction scores respectively corresponding to the at least two annotation dimensions, text annotation information corresponding to the target prediction data is generated.

[0021] Optionally, it also includes:

[0022] Determining a preset candidate text annotation model, and using the candidate text annotation model to process the target prediction data to obtain a candidate annotation text corresponding to the target prediction data;

[0023] Calculating text similarity between the candidate annotation text and the annotation text;

[0024] When the text similarity is greater than a preset similarity threshold, a step is performed to generate text annotation information corresponding to the target prediction data based on the annotation text and the prediction scores respectively corresponding to the at least two annotation dimensions.

[0025] Optionally, after the step of constructing a sample data group based on the target sample data, the target prediction data and the text annotation information is executed, the step further includes:

[0026] adding the sample data set to an intermediate sample data set;

[0027] When the intermediate sample data set meets the data set construction condition, performing data cleaning and format conversion on the intermediate sample data set;

[0028] A target sample data set is generated according to the data cleaning result and the format conversion result, wherein the target sample data set included in the target sample data set is used to perform the model training task and the model verification task.

[0029] Optionally, it also includes:

[0030] Determine the reward model and criticism model corresponding to the language model to be optimized;

[0031] Based on the target sample data set included in the target sample data set, the reward model and the criticism model are trained, and a target reward energy and a target criticism model are determined according to the training results;

[0032] The target reward model and the target criticism model are used to optimize the language model to be optimized until a target language model that meets the optimization stop condition is obtained.

[0033] According to a second aspect of an embodiment of this specification, a model optimization method is provided, including:

[0034] Determine the reward model and criticism model corresponding to the language model to be optimized;

[0035] Based on the target sample data set included in the target sample data set, the reward model and the criticism model are trained, and a target reward energy and a target criticism model are determined according to the training result, wherein the target sample data set included in the target sample data set is obtained by the above method;

[0036] The target reward model and the target criticism model are used to optimize the language model to be optimized until a target language model that meets the optimization stop condition is obtained.

[0037] According to a third aspect of an embodiment of this specification, there is provided a data processing device, including:

[0038] A determination module is configured to perform sampling in the plurality of data sets respectively and determine target sample data according to the sampling results;

[0039] A processing module is configured to input the target sample data into a plurality of language models for processing respectively, and obtain target prediction data outputted by the plurality of language models respectively;

[0040] An annotation module is configured to perform text annotation processing on each target prediction data using a preset text annotation model to obtain text annotation information corresponding to each target prediction data;

[0041] A construction module is configured to construct a sample data group based on the target sample data, the target prediction data and the text annotation information, wherein the sample data group is used to perform a model training task and a model verification task.

[0042] According to a fourth aspect of the embodiments of this specification, a model optimization device is provided, including:

[0043] A model determination module is configured to determine a reward model and a criticism model corresponding to the language model to be optimized;

[0044] A training model module is configured to train the reward model and the criticism model based on a target sample data set included in a target sample data set, and determine a target reward model and a target criticism model according to a training result, wherein the target sample data set included in the target sample data set is obtained by the above method;

[0045] The model optimization module is configured to optimize the language model to be optimized by using the target reward model and the target criticism model until a target language model that meets the optimization stop condition is obtained.

[0046] According to a fifth aspect of an embodiment of this specification, a computing device is provided, including:

[0047] Memory and processor;

[0048] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the above-mentioned data processing method or model optimization method are implemented.

[0049] According to the sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned data processing method or model optimization method are implemented.

[0050] According to the seventh aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions, which implement the steps of the above-mentioned data processing method or model optimization method when executed by a processor.

[0051] The data processing method provided in this embodiment can provide more abundant and diversified feedback data for model optimization for the language model, and can construct feedback data in combination with a text annotation model and multiple language models. Specifically, sampling can be performed in multiple data sets respectively to determine the target sample data according to the sampling results; and the sample diversity can be improved through multiple data sets. After that, the target sample data can be input into multiple language models for processing respectively, and the target prediction data output by multiple language models can be obtained; on this basis, the preset text annotation model can be used to perform text annotation processing on each target prediction data to obtain the text annotation information corresponding to each target prediction data; when constructing the text annotation information, in order to ensure the quality and data scale, multiple target prediction information can be output in combination with multiple language models, so as to effectively ensure the richness of the data, and then the sample data group can be constructed based on the target sample data, each target prediction data and the text annotation information corresponding to each target prediction data, so as to realize that in the application stage, the model training task and the model verification task can be performed according to the sample data group. To enhance the performance and alignment effect of the language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a schematic diagram of a data processing method provided by an embodiment of this specification;

[0053] Figure 2 is a flow chart of a data processing method provided by an embodiment of this specification;

[0054] Figure 3 is a flow chart of a model optimization method provided by an embodiment of this specification;

[0055] Figure 4 is a processing flow chart of a data processing method provided by an embodiment of this specification;

[0056] Figure 5 is a structural schematic diagram of a data processing device provided by an embodiment of this specification;

[0057] Figure 6 is a structural schematic diagram of a model optimization device provided by an embodiment of this specification;

[0058] Figure 7 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0059] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0060] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0061] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0062] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0063] First, the terms involved in one or more embodiments of this specification are explained.

[0064] LLMs, or Large Language Models, are a type of deep learning model in the field of natural language processing (NLP). The model is known for its large parameter size and powerful language understanding and generation capabilities. LLMs learn the complex patterns and rules of language by training on large-scale text datasets, so that they can perform various NLP tasks, including but not limited to text generation, text classification, sentiment analysis, question-answering systems, etc.

[0065] In this specification, a data processing method is provided. One or more embodiments of this specification also relate to a data processing device, a model optimization method, a model optimization device, a computing device, a computer-readable storage medium, and a computer program product, which are described in detail one by one in the following embodiments.

[0066] In practical applications, although the language model can be optimized through user feedback data, it is difficult to expand on a large scale due to time and budget constraints, and the lack of diversity. At the same time, different users have different subjective consciousness, which leads to inconsistent feedback data annotation standards. Therefore, an effective solution is urgently needed to solve the above problems.

[0067] See also Figure 1 As shown in the schematic diagram, the data processing method provided by this embodiment can be combined with a text annotation model and multiple language models to construct feedback data in order to provide more abundant and diverse feedback data for model optimization for the language model. Specifically, sampling can be performed in multiple data sets respectively to determine the target sample data according to the sampling results; and the sample diversity can be improved through multiple data sets. After that, the target sample data can be input into multiple language models for processing respectively to obtain the target prediction data output by multiple language models respectively; on this basis, the preset text annotation model can be used to perform text annotation processing on each target prediction data to obtain the text annotation information corresponding to each target prediction data; when constructing the text annotation information, in order to ensure the quality and data scale, multiple target prediction information can be output in combination with multiple language models, so as to effectively ensure the richness of the data, and then the sample data group can be constructed based on the target sample data, each target prediction data and the text annotation information corresponding to each target prediction data, so as to realize that in the application stage, the model training task and the model verification task can be performed according to the sample data group. To enhance the performance and alignment effect of the language model.

[0068] See also Figure 2 , Figure 2 A flow chart of a data processing method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0069] Step S202: Sampling is performed in multiple data sets respectively, and target sample data is determined according to the sampling results.

[0070] The data processing method provided in this embodiment can be applied to the scenario of constructing feedback data corresponding to a large language model under any business project, wherein the large language model that needs to be optimized for the constructed sample data group can be an LLMs model, and can also be other large language models with natural language processing capabilities. This embodiment does not make any limitations here.

[0071] Specifically, multiple data sets specifically refer to a collection of sample data provided by different data sources, and each data set contains the same type of sample data, but the types corresponding to each data set are different, thereby increasing the diversity of sample data set construction. Correspondingly, the target sample data specifically refers to any sample data that needs to be processed at the current moment after sampling processing.

[0072] Based on this, in order to provide more abundant and diversified feedback data for language model optimization, feedback data can be constructed by combining text annotation model and multiple language models. Specifically, sampling can be performed in multiple data sets respectively to determine the target sample data according to the sampling results; and sample diversity can be improved through multiple data sets. After that, the target sample data can be input into multiple language models for processing respectively to obtain target prediction data output by multiple language models respectively; on this basis, the preset text annotation model can be used to perform text annotation processing on each target prediction data to obtain the text annotation information corresponding to each target prediction data; when constructing text annotation information, in order to ensure the quality and data scale, multiple language models can be combined to output multiple target prediction information, so as to effectively ensure the richness of data, and then a sample data group can be constructed based on the target sample data, each target prediction data and the text annotation information corresponding to each target prediction data, so that in the application stage, model training tasks and model verification tasks can be performed according to the sample data group.

[0073] Furthermore, in order to build richer feedback data to improve the coverage of the optimized model, a stratified sampling method can be used to complete the sample sampling process. In this embodiment, the specific implementation method is as follows:

[0074] Determine multiple data sets, wherein the data types of the sample data contained in each data set are different; perform sampling in each data set to obtain multiple initial sample data, and construct an initial sample data set based on the multiple initial sample data; select target sample data from the initial sample data set.

[0075] Based on this, when performing sample sampling, multiple data sets can be determined first, and the data types of the sample data contained in each data set are different; thereafter, sampling can be performed separately in each data set. During the adoption process, sampling can be completed through a stratified sampling strategy to improve the sample richness. At this time, multiple initial sample data will be obtained, and then an initial sample data set can be constructed based on the multiple initial sample data; subsequently, any sample data in the initial sample data set can be selected as the target sample data to construct feedback data.

[0076] In practical applications, multiple data sets can be selected, such as TruthfulQA, FalseQA, Evol-Instruct, UltraChat, ShareGPT, etc., and this embodiment does not make any limitation here. When performing sample sampling, different types of sample data can be obtained from multiple data sources by using stratified sampling to ensure that the sample data can cover a variety of tasks and scenarios. In addition, sample data that is too long or inappropriate can be removed to ensure that the sample data is concise and clear.

[0077] For example, when constructing feedback data for optimizing the LLMs model, stratified sampling can be performed from multiple data sets such as TruthfulQA, FalseQA, Evol-Instruct, UltraChat, ShareGPT, etc. At this time, n sample data will be obtained according to the sampling results, and then by filtering the low-quality sample data of the n sample data, m target sample data will be obtained, so that feedback data can be constructed separately for the m target sample data in the future, thereby achieving the purpose of optimizing the LLMs model.

[0078] In practical applications, the target sample data may be question text, images, videos, etc. that are processed by the input model, and this embodiment does not impose any limitation thereto.

[0079] In summary, by adopting stratified sampling to construct the initial sample data set, the sample richness can be effectively improved, thereby covering more task scenarios and improving the generalization ability of the model.

[0080] Step S204: input the target sample data into a plurality of language models for processing respectively, and obtain target prediction data outputted by the plurality of language models respectively.

[0081] Specifically, after obtaining the target sample data as described above, in order to ensure the diversity of the feedback data, the target sample data can be processed in combination with multiple language models to obtain the target prediction data output by each language model, so as to subsequently combine multiple target prediction data to complete the feedback data construction.

[0082] Among them, multiple language models specifically refer to language models that can process the target sample data and output prediction results, such as GPT-4, gpt-3.5-turbo, Bard, etc. This embodiment does not make any restrictions here, and in the stage of building prediction data, the number of multiple language models selected can be set according to actual needs, and this embodiment does not make any restrictions here. Correspondingly, the target prediction data is the prediction result obtained after the model processes the target sample data. For example, if the target sample data is a question text, the target prediction data is the answer text output by the language model. And because different models have different capabilities, multiple different prediction data can be obtained.

[0083] Furthermore, in order to output different prediction information through multiple language models and then obtain different text annotation information in the future to improve the richness of feedback data, different prompt information corresponding to different language models can be used to update the sample data and then generate prediction information. In this embodiment, the specific implementation method is as follows:

[0084] Select multiple language models for the target sample data, and determine the prompt information corresponding to each language model; wherein the processing of the target sample data by any one of the multiple language models includes: updating the target sample data using the prompt information corresponding to the language model, generating model input context information according to the update result; inputting the model input context information into the language model for processing to obtain target prediction data.

[0085] Specifically, the prompt information specifically refers to the prompt words corresponding to different language models. Correspondingly, the model input context information specifically refers to the text information obtained after updating the target sample data using the prompt information, which is used for inputting the model to generate predictive data.

[0086] Based on this, when selecting multiple language models to generate prediction data, multiple language models can be selected according to the target sample data, and the prompt information corresponding to each language model needs to be determined; after the target sample data is updated through different prompt information, different prediction data can be given by inputting different language models, so as to facilitate the subsequent construction of diverse feedback data; for the processing of the target sample data by any one of the multiple language models, the target sample data can be updated using the prompt information corresponding to the language model, so as to generate model input context information according to the update result; thereafter, the model input context information can be input into the language model for processing, so as to obtain the target prediction data output by the language model; and so on, any language model can be processed in this way, so as to obtain multiple prediction data for subsequent use.

[0087] In practical applications, in order to avoid the problem of consistency in text style and quality of the prediction data generated for sample data, different multilingual models can be randomly selected to output the target prediction data corresponding to the target sample data, and different principles and prompt words can be introduced to update the sample data to induce the language model to output target prediction data of different styles and qualities.

[0088] Continuing with the above example, after obtaining m target sample data (this embodiment is illustrated as m=2), two language models can be selected for target sample data 1 and target sample data 2, namely language model a and language model b; then the prompt words corresponding to language models a and b can be determined, and the target sample data 1 is updated to target sample data 1a and 1b, and the target sample data 2 is updated to target sample data 2a and 2b; further, by inputting the target sample data 1a and the target sample data 2a into the language model a for processing, the target prediction data 1a corresponding to the target sample data 1 and the target prediction data 2a corresponding to the target sample data 2 are obtained, and at the same time, the target sample data 1b and the target sample data 2b are input into the language model b for processing, and the target prediction data 1b corresponding to the target sample data 1 and the target prediction data 2b corresponding to the target sample data 2 are obtained, wherein the style and quality corresponding to each prediction data are different, so as to facilitate subsequent use.

[0089] In summary, by introducing prompt information to update the sample data, the language model can output responses of different quality and style for the sample data, thereby enabling the subsequent use of diverse data to complete the feedback data construction.

[0090] Step S206: perform text annotation processing on each target prediction data using a preset text annotation model to obtain text annotation information corresponding to each target prediction data.

[0091] Specifically, after obtaining multiple prediction data corresponding to the target sample data as mentioned above, in order to improve the efficiency of feedback data construction and reduce costs at the same time, a preset text annotation model can be used to perform text annotation processing on each target prediction data, and then the text annotation information corresponding to each target prediction data can be obtained, thereby realizing automated text annotation through the text annotation model to improve the efficiency of feedback data construction.

[0092] Among them, the text annotation model specifically refers to a model that can generate feedback annotations for the target prediction data after inputting the target prediction data corresponding to the target sample data, which can be implemented using GPT-4. Correspondingly, the text annotation information specifically refers to the annotation information obtained after the text annotation model performs text annotation processing on the target prediction data, which is used to simulate the user's real feedback data, thereby facilitating the subsequent optimization of the language model.

[0093] Furthermore, when constructing text annotation information, the annotation text and the predicted score can be combined to construct it, so as to facilitate the subsequent screening of high-quality feedback data for use. In this embodiment, the specific implementation method is as follows:

[0094] The target sample data and the target prediction data are input into a preset text annotation model, the target prediction data is processed in at least two annotation dimensions using the text annotation model, and the prediction scores corresponding to the annotation text and the at least two annotation dimensions are determined according to the processing results; based on the annotation text and the prediction scores corresponding to the at least two annotation dimensions, text annotation information corresponding to the target prediction data is generated.

[0095] Specifically, the prediction score refers to the score obtained by the text annotation model after scoring the target prediction data in at least two annotation dimensions, wherein the at least two annotation dimensions may include instruction compliance dimension, authenticity dimension, honesty dimension and usefulness dimension, and then the availability of the target prediction data is reflected through the prediction scores corresponding to multiple dimensions, so as to facilitate the subsequent division of positive samples and negative samples to complete model optimization.

[0096] Based on this, for any target prediction data, the target sample data and the target prediction data can be input into a preset text annotation model, and the text annotation model can be used to process the target prediction data in at least two annotation dimensions to determine the prediction scores corresponding to the annotation text and at least two annotation dimensions according to the processing results; thereafter, based on the annotation text and the prediction scores corresponding to at least two annotation dimensions, text annotation information corresponding to the target prediction data can be generated to reflect the availability of the target prediction data so that it can be used for subsequent model optimization.

[0097] Continuing with the above example, after processing the target sample data 1 and the target sample data 2 through multiple language models, the target prediction data 1a and the target prediction data 1b corresponding to the target sample data 1, and the target prediction data 2a and the target prediction data 2b corresponding to the target sample data 2 will be obtained; on this basis, the text annotation model can be used to score the target prediction data in the above four dimensions and generate detailed text annotations. After the text annotation model scores the prediction data corresponding to each sample data and generates annotations, the prediction score sa1 corresponding to the target prediction data 1a and the annotation text 1ac will be obtained; the prediction score sb1 corresponding to the target prediction data 1b and the annotation text 1bc will be obtained; the prediction score sa2 corresponding to the target prediction data 2a and the annotation text 2ac will be obtained; the prediction score sb2 corresponding to the target prediction data 2b and the annotation text 2bc will be obtained; subsequently, the above information can be combined to screen high-quality prediction data and annotation text to construct feedback data for training the language model.

[0098] In summary, by combining the prediction score and the annotation text to construct text annotation information, the prototype construction of the sample data group can be realized, which is convenient for subsequent model optimization and use.

[0099] In addition, in order to ensure the quality of the annotated text, the quality can also be improved by predicting the candidate text annotation model. In this embodiment, the specific implementation method is as follows:

[0100] Determine a preset candidate text annotation model, and use the candidate text annotation model to process the target prediction data to obtain the candidate annotation text corresponding to the target prediction data; calculate the text similarity between the candidate annotation text and the annotation text; when the text similarity is greater than a preset similarity threshold, execute the steps of generating text annotation information corresponding to the target prediction data based on the prediction scores corresponding to the annotation text and the at least two annotation dimensions respectively.

[0101] Specifically, the candidate text annotation model specifically refers to a text annotation model that has the same capabilities as the text annotation model but a different model structure, and is used to generate candidate annotation texts for prediction data, so as to screen out high-quality annotation texts for subsequent sample data group construction after comparison with the annotation texts. Correspondingly, the candidate annotation texts are the annotation texts output by the candidate text annotation model. Correspondingly, the text similarity specifically refers to the score for calculating the degree of similarity between the candidate annotation texts and the annotation texts, which can be accomplished by calculating text distance and cosine similarity, and this embodiment does not make any limitation here.

[0102] Based on this, in order to obtain high-quality annotated text, for any target prediction data, a candidate text annotation model can also be used for processing. At this time, the preset candidate text annotation model can be determined first, and then the candidate text annotation model can be used to process the target prediction data to obtain the candidate annotation text corresponding to the target prediction data; on this basis, the text similarity between the candidate annotation text and the annotation text can be calculated; when the text similarity is greater than the preset similarity threshold, it means that the annotation texts output by the two different text annotation models are highly similar, and then it can be said that the annotation text is a high-quality text, so the step of generating the text annotation information corresponding to the target prediction data based on the prediction scores corresponding to the annotation text and the at least two annotation dimensions can be continued.

[0103] In addition, when the text similarity is less than or equal to the preset similarity threshold, it indicates the similarity angle of the annotation texts output by the two different text annotation models, and further indicates that the annotation text output by one of the models is not accurate enough. In order to obtain accurate annotation texts, the two models can be used to generate annotation texts again, until an annotation text with a similarity greater than the threshold is obtained at a certain moment, and subsequent processing can be performed. In addition, if annotation text with a higher similarity cannot be obtained for a long time, a third text annotation model can be introduced to make a decision, thereby ensuring the accuracy of the annotation text and improving the quality of the text annotation information.

[0104] In summary, by combining the candidate text annotation model to construct the annotation text, it is possible to screen high-quality annotation texts, thereby effectively improving the quality of the sample data set to achieve a better optimization model.

[0105] Step S208, constructing a sample data group based on the target sample data, the target prediction data and the text annotation information, wherein the sample data group is used to perform a model training task and a model verification task.

[0106] Specifically, after obtaining the target sample data, the target prediction data output by each language model, and the text annotation information corresponding to each target prediction data, a sample data group can be constructed by combining the target sample data, the target prediction data and the text annotation information to realize the subsequent combination of the sample data group to perform model training tasks and model verification tasks.

[0107] The sample data set is the feedback data used to optimize the language model. Correspondingly, the model training task specifically refers to the task of training the model using the sample data set, and the model verification task specifically refers to the task of verifying the model using the sample data set.

[0108] Furthermore, after obtaining the sample data set, in order to improve the quality, data cleaning and format conversion can be performed so that it can be directly reused in the application stage. In this embodiment, the specific implementation method is as follows:

[0109] The sample data group is added to an intermediate sample data set; if the intermediate sample data set meets the data set construction conditions, data cleaning and format conversion are performed on the intermediate sample data set; a target sample data set is generated according to the data cleaning results and the format conversion results, wherein the target sample data group included in the target sample data set is used to perform the model training task and the model verification task.

[0110] Specifically, the intermediate sample data set specifically refers to the data set obtained by constructing the sample data set. Accordingly, data cleaning specifically refers to the processing operation of removing duplicate and low-quality sample data sets in the intermediate sample data set. Accordingly, format conversion specifically refers to the processing operation of updating the sample data sets contained in the intermediate sample data set to a unified data structure.

[0111] Based on this, in order to be able to use high-quality data for model training and model verification, after obtaining the sample data group, the sample data group can be added to the intermediate sample data set; when the intermediate sample data set meets the data set construction conditions, it means that the sample data group contained in the intermediate sample data set can be used for subsequent tasks. At this time, data cleaning and format conversion can be performed on the intermediate sample data set first; redundant data can be deleted and the data format can be unified, and then the target sample data set can be generated according to the data cleaning results and format conversion results. Subsequently, the model training task and model verification task can be performed according to the target sample data set contained in the target sample data set.

[0112] In summary, by cleaning and formatting the intermediate sample data set, a pure and high-quality sample data set can be obtained with the same data format, which is more convenient for subsequent model training and evaluation.

[0113] In addition, after obtaining a high-quality target sample data set, in order to simulate a real model optimization scenario, the reward model and the criticism model can be trained using the data first, and then the language model can be optimized by combining the trained models. In this embodiment, the specific implementation method is as follows:

[0114] Determine a reward model and a criticism model corresponding to the language model to be optimized; train the reward model and the criticism model based on the target sample data set included in the target sample data set, and determine a target reward model and a target criticism model according to the training results; optimize the language model to be optimized using the target reward model and the target criticism model until a target language model that meets the optimization stop condition is obtained.

[0115] Specifically, the reward model and the criticism model specifically refer to auxiliary models used to optimize the language model to be optimized, wherein the reward model is used to evaluate and improve the prediction quality of the language model to be optimized, and the criticism model is used to provide detailed improvement suggestions, so that the language model to be optimized can achieve the purpose of self-learning. Correspondingly, the optimization stop condition specifically refers to the condition for stopping the optimization of the language model to be optimized, such as the number of iterations, the loss value comparison condition, or the verification set verification condition, etc., which is not limited in this embodiment.

[0116] Based on this, in order to effectively optimize the language model to be optimized, the reward model and the criticism model corresponding to the language model to be optimized can be determined first; at this time, the reward model and the criticism model can be trained based on the target sample data group contained in the target sample data set, and the target reward model and the target criticism model can be determined according to the training results; on this basis, the target reward model and the target criticism model can be used to optimize the language model to be optimized, so as to adjust the model to be optimized to the target language model that meets the optimization stop conditions, so as to facilitate application in actual business scenarios.

[0117] Continuing with the above example, after obtaining the prediction data, prediction scores, and annotation text corresponding to target sample data 1 and target sample data 2, respectively, an intermediate sample data set can be constructed by combining the data of each dimension. After that, a high-quality target sample data set will be obtained by cleaning and formatting the data set. On this basis, the target sample data set can be used to train the reward model and criticism model of the LLMs model, so that the reward model can improve the output quality of the LLMs model, and the criticism model can provide detailed model improvement suggestions. By optimizing the LLMs model on this basis, a target LLMs model that meets the usage conditions can be obtained. In the application stage, if the user enters the question text "What is the capital of place A", the model can feedback the answer text "City B".

[0118] The data processing method provided in this embodiment can provide more abundant and diversified feedback data for model optimization for the language model, and can construct feedback data in combination with a text annotation model and multiple language models. Specifically, sampling can be performed in multiple data sets respectively to determine the target sample data according to the sampling results; and the sample diversity can be improved through multiple data sets. After that, the target sample data can be input into multiple language models for processing respectively, and the target prediction data output by multiple language models can be obtained; on this basis, the preset text annotation model can be used to perform text annotation processing on each target prediction data to obtain the text annotation information corresponding to each target prediction data; when constructing the text annotation information, in order to ensure the quality and data scale, multiple target prediction information can be output in combination with multiple language models, so as to effectively ensure the richness of the data, and then the sample data group can be constructed based on the target sample data, each target prediction data and the text annotation information corresponding to each target prediction data, so as to realize that in the application stage, the model training task and the model verification task can be performed according to the sample data group. To enhance the performance and alignment effect of the language model.

[0119] See also Figure 3 , Figure 3 A flow chart of a model optimization method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0120] Step S302, determining a reward model and a criticism model corresponding to the language model to be optimized;

[0121] Step S304, training the reward model and the critic model based on the target sample data set included in the target sample data set, and determining a target reward energy and a target critic model according to the training result, wherein the target sample data set included in the target sample data set is obtained by the above method;

[0122] Step S306, optimizing the language model to be optimized by using the target reward model and the target criticism model until a target language model that meets the optimization stop condition is obtained.

[0123] For the description of the model optimization method provided in this embodiment, reference can be made to the description of the data processing method in the above embodiment, and this embodiment will not be elaborated in detail here.

[0124] The following combination Figure 4 , taking the application of the data processing method provided in this specification in a large language model scenario as an example, the data processing method is further described. Figure 4 A processing flow chart of a data processing method provided by an embodiment of the present specification is shown, which specifically includes the following steps.

[0125] Step S402: determining a plurality of data sets, wherein the data types of the sample data contained in each data set are different.

[0126] Step S404 , sampling is performed in each data set to obtain a plurality of initial sample data, and an initial sample data set is constructed based on the plurality of initial sample data.

[0127] Step S406: Select target sample data from the initial sample data set.

[0128] Step S408: input the target sample data into a plurality of language models for processing respectively, and obtain target prediction data outputted by the plurality of language models respectively.

[0129] Step S410: perform text annotation processing on each target prediction data using a preset text annotation model to obtain text annotation information corresponding to each target prediction data.

[0130] Among them, determining the text annotation information of any target prediction data among the target prediction data corresponding to the multiple language models respectively includes: inputting the target sample data and the target prediction data into a preset text annotation model, using the text annotation model to process the target prediction data in at least two annotation dimensions, and determining the annotation text and the prediction scores corresponding to the at least two annotation dimensions respectively according to the processing results; based on the annotation text and the prediction scores corresponding to the at least two annotation dimensions respectively, generating the text annotation information corresponding to the target prediction data.

[0131] Step S412: construct a sample data set based on the target sample data, the target prediction data and the text annotation information, and add the sample data set to the intermediate sample data set.

[0132] Step S414: When the intermediate sample data set meets the data set construction condition, data cleaning and format conversion are performed on the intermediate sample data set.

[0133] Step S416, generating a target sample data set according to the data cleaning results and the format conversion results.

[0134] Step S418, when the model optimization task is triggered, determining the reward model and criticism model corresponding to the language model to be optimized.

[0135] Step S420, training the reward model and the criticism model based on the target sample data set included in the target sample data set, and determining the target reward model and the target criticism model according to the training results.

[0136] Step S422, optimizing the language model to be optimized by using the target reward model and the target criticism model until a target language model that meets the optimization stop condition is obtained.

[0137] The data processing method provided in this embodiment can provide more abundant and diversified feedback data for model optimization for the language model, and can construct feedback data in combination with a text annotation model and multiple language models. Specifically, sampling can be performed in multiple data sets respectively to determine the target sample data according to the sampling results; and the sample diversity can be improved through multiple data sets. After that, the target sample data can be input into multiple language models for processing respectively, and the target prediction data output by multiple language models can be obtained; on this basis, the preset text annotation model can be used to perform text annotation processing on each target prediction data to obtain the text annotation information corresponding to each target prediction data; when constructing the text annotation information, in order to ensure the quality and data scale, multiple target prediction information can be output in combination with multiple language models, so as to effectively ensure the richness of the data, and then the sample data group can be constructed based on the target sample data, each target prediction data and the text annotation information corresponding to each target prediction data, so as to realize that in the application stage, the model training task and the model verification task can be performed according to the sample data group. To enhance the performance and alignment effect of the language model.

[0138] Corresponding to the above method embodiment, this specification also provides a data processing device embodiment, Figure 5 FIG. 1 is a schematic diagram showing the structure of a data processing device provided by an embodiment of the present specification. Figure 5 As shown, the device comprises:

[0139] The determination module 502 is configured to perform sampling in the plurality of data sets respectively and determine the target sample data according to the sampling results;

[0140] The processing module 504 is configured to input the target sample data into multiple language models for processing respectively, and obtain target prediction data outputted by the multiple language models respectively;

[0141] Annotation module 506, configured to perform text annotation processing on each target prediction data using a preset text annotation model to obtain text annotation information corresponding to each target prediction data;

[0142] The construction module 508 is configured to construct a sample data group based on the target sample data, the target prediction data and the text annotation information, wherein the sample data group is used to perform a model training task and a model verification task.

[0143] In an optional embodiment, the determining module 502 is further configured to:

[0144] Determine multiple data sets, wherein the data types of sample data contained in each data set are different; perform sampling in each data set to obtain multiple initial sample data, and construct an initial sample data set based on the multiple initial sample data; select target sample data from the initial sample data set.

[0145] In an optional embodiment, the device further comprises:

[0146] The information determination module is configured to select multiple language models for the target sample data and determine the prompt information corresponding to each language model; wherein the processing of the target sample data by any one of the multiple language models includes: updating the target sample data using the prompt information corresponding to the language model, generating model input context information according to the update result; inputting the model input context information into the language model for processing to obtain target prediction data.

[0147] In an optional embodiment, the annotation module 506 is further configured to:

[0148] The target sample data and the target prediction data are input into a preset text annotation model, the target prediction data is processed in at least two annotation dimensions using the text annotation model, and the prediction scores corresponding to the annotation text and the at least two annotation dimensions are determined according to the processing results. Based on the annotation text and the prediction scores corresponding to the at least two annotation dimensions, text annotation information corresponding to the target prediction data is generated.

[0149] In an optional embodiment, the device further comprises:

[0150] The similarity calculation module is configured to determine a preset candidate text annotation model, and use the candidate text annotation model to process the target prediction data to obtain the candidate annotation text corresponding to the target prediction data; calculate the text similarity between the candidate annotation text and the annotation text; and when the text similarity is greater than a preset similarity threshold, execute the steps of generating text annotation information corresponding to the target prediction data based on the prediction scores corresponding to the annotation text and the at least two annotation dimensions respectively.

[0151] In an optional embodiment, the device further comprises:

[0152] A building set module is configured to add the sample data set to an intermediate sample data set;

[0153] When the intermediate sample data set meets the data set construction condition, performing data cleaning and format conversion on the intermediate sample data set;

[0154] A target sample data set is generated according to the data cleaning result and the format conversion result, wherein the target sample data set included in the target sample data set is used to perform the model training task and the model verification task.

[0155] In an optional embodiment, the device further comprises:

[0156] The model optimization module is configured to determine a reward model and a criticism model corresponding to a language model to be optimized; based on a target sample data set included in the target sample data set, the reward model and the criticism model are trained, and a target reward model and a target criticism model are determined according to the training results; and the language model to be optimized is optimized using the target reward model and the target criticism model until a target language model that meets the optimization stop condition is obtained.

[0157] The data processing device provided in this embodiment can combine a text annotation model and multiple language models to construct feedback data in order to provide more abundant and diverse feedback data for the language model for model optimization. Specifically, sampling can be performed in multiple data sets respectively to determine the target sample data according to the sampling results; and the sample diversity can be improved through multiple data sets. After that, the target sample data can be input into multiple language models for processing respectively to obtain the target prediction data output by multiple language models respectively; on this basis, the preset text annotation model can be used to perform text annotation processing on each target prediction data to obtain the text annotation information corresponding to each target prediction data; when constructing the text annotation information, in order to ensure the quality and data scale, multiple target prediction information can be output in combination with multiple language models, so as to effectively ensure the richness of the data, and then a sample data group can be constructed based on the target sample data, each target prediction data and the text annotation information corresponding to each target prediction data, so as to realize that in the application stage, the model training task and the model verification task can be performed according to the sample data group. To enhance the performance and alignment effect of the language model.

[0158] The above is a schematic scheme of a data processing device of this embodiment. It should be noted that the technical scheme of the data processing device and the technical scheme of the above data processing method belong to the same concept, and the details of the technical scheme of the data processing device that are not described in detail can be referred to the description of the technical scheme of the above data processing method.

[0159] Corresponding to the above method embodiment, this specification also provides a model optimization device embodiment, Figure 6FIG. 1 shows a schematic diagram of a model optimization device provided by an embodiment of the present specification. Figure 6 As shown, the device comprises:

[0160] A model determination module 602 is configured to determine a reward model and a criticism model corresponding to the language model to be optimized;

[0161] A training model module 604 is configured to train the reward model and the critic model based on a target sample data set included in a target sample data set, and determine a target reward model and a target critic model according to a training result, wherein the target sample data set included in the target sample data set is obtained by the above method;

[0162] The model optimization module 606 is configured to optimize the language model to be optimized using the target reward model and the target criticism model until a target language model that meets the optimization stop condition is obtained.

[0163] The above is a schematic scheme of a model optimization device of this embodiment. It should be noted that the technical scheme of the model optimization device and the technical scheme of the above-mentioned model optimization method belong to the same concept, and the details not described in detail in the technical scheme of the model optimization device can be referred to the description of the technical scheme of the above-mentioned model optimization method.

[0164] Figure 7 The block diagram of a computing device 700 according to an embodiment of the present specification is shown. The components of the computing device 700 include but are not limited to a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and the database 750 is used to store data.

[0165] The computing device 700 also includes an access device 740 that enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0166] In one embodiment of the present specification, the above components of the computing device 700 and Figure 7 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 7 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0167] The computing device 700 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 700 may also be a mobile or stationary server.

[0168] The processor 720 is used to execute the following computer executable instructions, which, when executed by the processor, implement the steps of the above-mentioned data processing method or model optimization method.

[0169] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned data processing method or model optimization method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the above-mentioned data processing method or model optimization method.

[0170] An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned data processing method or model optimization method.

[0171] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the above-mentioned data processing method or model optimization method belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the above-mentioned data processing method or model optimization method.

[0172] An embodiment of the present specification also provides a computer program product, including a computer program or instructions, which implement the steps of the above-mentioned data processing method or model optimization method when executed by a processor.

[0173] The above is a schematic scheme of a computer program product of this embodiment. It should be noted that the technical scheme of the computer program product and the technical scheme of the above-mentioned data processing method or model optimization method belong to the same concept, and the details not described in detail in the technical scheme of the computer program product can be referred to the description of the technical scheme of the above-mentioned data processing method or model optimization method.

[0174] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0175] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0176] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0177] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0178] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can understand and use this specification well.

Claims

1. A data processing method, characterized in that: include: Sampling is performed in multiple data sets respectively, and target sample data is determined according to the sampling results; Inputting the target sample data into a plurality of language models for processing respectively, and obtaining target prediction data outputted by the plurality of language models respectively; Perform text annotation processing on each target prediction data using a preset text annotation model to obtain text annotation information corresponding to each target prediction data; A sample data group is constructed based on the target sample data, the target prediction data and the text annotation information, wherein the sample data group is used to perform a model training task and a model verification task.

2. The data processing method according to claim 1, characterized in that: The sampling is performed in the plurality of data sets respectively, and the target sample data is determined according to the sampling results, including: Determine a plurality of data sets, wherein the data types of the sample data contained in each data set are different; Sampling is performed in each data set to obtain a plurality of initial sample data, and an initial sample data set is constructed based on the plurality of initial sample data; Select target sample data from the initial sample data set.

3. The data processing method according to claim 1, characterized in that: Before the step of inputting the target sample data into a plurality of language models for processing respectively and obtaining the target prediction data outputted by the plurality of language models respectively is executed, the method further includes: Selecting multiple language models for the target sample data, and determining prompt information corresponding to each language model; The processing of the target sample data by any one of the multiple language models includes: The target sample data is updated using prompt information corresponding to the language model, and model input context information is generated according to the update result; The model input context information is input into the language model for processing to obtain target prediction data.

4. The data processing method according to claim 1, characterized in that: Determining text annotation information of any one of the target prediction data respectively corresponding to the multiple language models includes: Inputting the target sample data and the target prediction data into a preset text annotation model, processing the target prediction data in at least two annotation dimensions using the text annotation model, and determining the prediction scores corresponding to the annotation text and the at least two annotation dimensions respectively according to the processing results; Based on the annotation text and the prediction scores respectively corresponding to the at least two annotation dimensions, text annotation information corresponding to the target prediction data is generated.

5. The data processing method according to claim 4, characterized in that: Also includes: Determining a preset candidate text annotation model, and using the candidate text annotation model to process the target prediction data to obtain a candidate annotation text corresponding to the target prediction data; Calculating text similarity between the candidate annotation text and the annotation text; When the text similarity is greater than a preset similarity threshold, a step is performed to generate text annotation information corresponding to the target prediction data based on the annotation text and the prediction scores respectively corresponding to the at least two annotation dimensions.

6. The data processing method according to claim 1, characterized in that: After the step of constructing a sample data group based on the target sample data, the target prediction data and the text annotation information is executed, the method further includes: adding the sample data set to an intermediate sample data set; When the intermediate sample data set meets the data set construction condition, performing data cleaning and format conversion on the intermediate sample data set; A target sample data set is generated according to the data cleaning result and the format conversion result, wherein the target sample data set included in the target sample data set is used to perform the model training task and the model verification task.

7. The data processing method according to claim 6, characterized in that: Also includes: Determine the reward model and criticism model corresponding to the language model to be optimized; Based on the target sample data set included in the target sample data set, the reward model and the criticism model are trained, and a target reward energy and a target criticism model are determined according to the training results; The target reward model and the target criticism model are used to optimize the language model to be optimized until a target language model that meets the optimization stop condition is obtained.

8. A model optimization method, characterized in that: include: Determine the reward model and criticism model corresponding to the language model to be optimized; Based on the target sample data set included in the target sample data set, the reward model and the criticism model are trained, and a target reward energy and a target criticism model are determined according to the training result, wherein the target sample data set included in the target sample data set is obtained by the method according to any one of claims 1 to 7; The target reward model and the target criticism model are used to optimize the language model to be optimized until a target language model that meets the optimization stop condition is obtained.

9. A data processing device, characterized in that: include: A determination module is configured to perform sampling in the plurality of data sets respectively and determine target sample data according to the sampling results; A processing module is configured to input the target sample data into a plurality of language models for processing respectively, and obtain target prediction data outputted by the plurality of language models respectively; An annotation module is configured to perform text annotation processing on each target prediction data using a preset text annotation model to obtain text annotation information corresponding to each target prediction data; A construction module is configured to construct a sample data group based on the target sample data, the target prediction data and the text annotation information, wherein the sample data group is used to perform a model training task and a model verification task.

10. A model optimization device, characterized in that: include: A model determination module is configured to determine a reward model and a criticism model corresponding to the language model to be optimized; A training model module, configured to train the reward model and the criticism model based on a target sample data set included in a target sample data set, and determine a target reward model and a target criticism model according to the training result, wherein the target sample data set included in the target sample data set is obtained by the method according to any one of claims 1 to 7; The model optimization module is configured to optimize the language model to be optimized by using the target reward model and the target criticism model until a target language model that meets the optimization stop condition is obtained.

11. A computing device, characterized in that: include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the method described in any one of claims 1 to 8 are implemented.

12. A computer-readable storage medium, characterized in that: It stores computer executable instructions, which, when executed by a processor, implement the steps of the method described in any one of claims 1 to 8.

13. A computer program product, characterized in that The method comprises a computer program or an instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 8.

Images