Model evaluation system, data processing method and device, equipment and medium

Abstract

The invention relates to the technical field of data processing, in particular to a model evaluation system, a data processing method and device, equipment and a medium, and aims to effectively evaluate the performance of a model. The system comprises a data processing unit used for calling at least one first model to execute a data processing task to obtain at least one data processing result, and the at least one first model is a different model; the result evaluation unit is used for evaluating the at least one data processing result through a second model to obtain at least one evaluation result and outputting the evaluation result to a user page, and the evaluation result is used for comparing differences between the data processing results of different first models; and the user feedback acquisition unit is used for acquiring feedback information of the user on the evaluation result, so that the second model optimizes the evaluation logic according to the feedback information.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and specifically, to a model evaluation system, a data processing method, an apparatus, a device, and a medium. Background Art

[0002] At present, there are a large number of artificial intelligence models for performing data processing tasks. They can generate corresponding data processing results according to the needs put forward by users. The accuracy and quality of the data processing results generated by the model need to be evaluated. In related technologies, the evaluation of the data processing results of the model is generally carried out through manual evaluation or using simple rules.

[0003] In the related art, the method used to evaluate the data processing results of the model is relatively simple and cannot effectively evaluate the actual effect of each model. Summary of the invention

[0004] The embodiments of the present application provide a model evaluation system, data processing method, device, equipment and medium, which are intended to effectively evaluate the data processing results of the model.

[0005] A first aspect of an embodiment of the present application provides a model evaluation system, the system comprising: A data processing unit, configured to call at least one first model to perform a data processing task and obtain at least one data processing result, wherein the at least one first model is a different model; A result evaluation unit, used to evaluate the at least one data processing result respectively through the second model to obtain at least one evaluation result, and output the evaluation result to a user page, wherein the evaluation result is used to compare the differences between the data processing results of different first models; A user feedback acquisition unit is used to obtain user feedback information on the evaluation result, so that the second model optimizes the evaluation logic according to the feedback information.

[0006] Optionally, the result evaluation unit comprises: The dimension evaluation subunit is used to evaluate the data processing result from at least one preset dimension to obtain the evaluation result.

[0007] Optionally, the user feedback acquisition unit includes at least one of the following: A first user feedback acquisition subunit is used to acquire the evaluation selection made by the user on each of the evaluation results, wherein the evaluation selection includes any one of the following: over-evaluation, normal evaluation, and under-evaluation; The second user feedback acquisition subunit is used to acquire the user's evaluation content for each of the evaluation results, where the evaluation content includes a score and evaluation information for each of the evaluation results.

[0008] Optionally, the system further comprises: A model loading unit, used to load the corresponding first model according to the model information input by the user; A state setting unit, used to set the use state of the first model, wherein the use state includes an enabled state and a disabled state; A model information display unit, used to display model information of the first model; A parameter configuration unit is used to configure parameters of the first model according to the model configuration parameters.

[0009] Optionally, the system further comprises: A file receiving unit, configured to receive a task file through a first interface, wherein the task file includes task data corresponding to the data processing task; A result file acquisition unit is used to output a result file corresponding to the data processing task through a second interface, wherein the result file includes the data processing result of the first model and the evaluation result of the second model.

[0010] Optionally, the system further comprises: An evaluation result display unit, used to display the evaluation score corresponding to each of the first models and the evaluation reason corresponding to the evaluation score; an image generating unit, configured to calculate an average score of the evaluation scores corresponding to all the first models when all the first models have completed the data processing task, generate a corresponding score distribution graph, and mark the average score in the score distribution graph; An evaluation history display unit is used to display the data processing results corresponding to each of the data processing tasks executed by the first model, and the evaluation records corresponding to the data processing results, wherein the evaluation records include the task content of the data processing task, the evaluation results corresponding to the data processing results, and the reasons for the evaluation.

[0011] A second aspect of an embodiment of the present application provides a data processing method, which is applied to a model evaluation system, comprising: Receive task data corresponding to the data processing task; Execute the data processing task by at least one first model to obtain at least one data processing result; inputting the at least one data processing result into a second model; Evaluate the data processing result from at least one preset dimension using the second model to obtain at least one evaluation result; According to the evaluation result, a target data processing result is determined from the at least one data processing result.

[0012] Optionally, before performing the data processing task by using at least one first model to obtain at least one data processing result, the method further includes: receiving model information of the first model to be loaded; In a case where the model address included in the model information is a local path, acquiring the first model from a storage location corresponding to the local path; In a case where the model address included in the model information is a program interface address, downloading the first model through the program interface address; According to the model parameter type included in the model information, inputting the corresponding model parameter value into the first model; Set the first model to an enabled state.

[0013] Optionally, the method further comprises: storing the data processing result corresponding to the first model, the data processing result and the corresponding evaluation reason in the evaluation record corresponding to the first model; When a request for viewing the evaluation record corresponding to the first model is received, the evaluation record is displayed.

[0014] Optionally, the method further comprises: When a re-evaluation instruction is received, the at least one data processing result is re-evaluated by using the second model.

[0015] Optionally, evaluating the data processing result from at least one preset dimension using the second model to obtain at least one evaluation result includes: Using a preset format matching rule, scoring the data processing result from a format dimension to obtain a format scoring result; By calculating the consistency score between the content of the data processing result and the theme, the data processing result is scored from the theme expression dimension to obtain a theme expression scoring result; Sorting the data processing results according to the degree of perplexity, and scoring the data processing results from the dimension of fluency to obtain a fluency scoring result; Using a preset repetition evaluation rule, the data processing result is scored from the repetition dimension to obtain a repetition scoring result; Using a preset rhyme judgment rule, scoring the data processing result from a rhythm dimension to obtain a rhythm scoring result; Using a preset rhetoric judgment method, scoring the data processing results from the rhetoric skill dimension to obtain a rhetoric scoring result; The format scoring result, the theme expression scoring result, the fluency scoring result, the repetition scoring result, the rhythm scoring result, and the rhetoric scoring result are weighted calculated according to a preset weight ratio to obtain the data processing result.

[0016] A third aspect of an embodiment of the present application provides a data processing device, which is applied to a model evaluation system, including: A task data receiving module, used for receiving task data corresponding to a data processing task; A data processing result acquisition module, used to execute a data processing task through the at least one first model to obtain at least one data processing result; A data processing result input module, used for inputting the at least one data processing result into the second model; a data processing result evaluation module, configured to evaluate the data processing result from at least one preset dimension using the second model to obtain at least one evaluation result; A data processing result determination module is used to determine a target data processing result from the at least one data processing result according to the evaluation result.

[0017] Optionally, the device further comprises: A model information receiving module, used to receive model information of at least one first model to be loaded; A first model acquisition submodule, configured to acquire the first model from a storage location corresponding to the local path when the model address is a local path; A second model acquisition submodule, configured to download the first model through the program interface address when the model address is a program interface address; A parameter input submodule, used for inputting the corresponding model parameter value into the first model according to the model parameter type; The model activation module is used to set the first model to an activated state when the first model is loaded.

[0018] Optionally, the device further comprises: An evaluation record module, used for storing the data processing result corresponding to the first model, the data processing result and the corresponding evaluation reason in an evaluation record corresponding to the first model; The evaluation record display module is used to display the evaluation record when receiving a request to view the evaluation record corresponding to the first model.

[0019] Optionally, the device further comprises: A re-evaluation module is used to re-evaluate the at least one data processing result by using the second model when a re-evaluation instruction is received.

[0020] Optionally, the data processing result evaluation module includes: A first evaluation submodule is used to score the data processing result from the format dimension using a preset format matching rule to obtain a format scoring result; The second evaluation submodule is used to score the data processing result from the dimension of theme expression by calculating the consistency score between the content of the data processing result and the theme, so as to obtain a theme expression scoring result; A third evaluation submodule is used to sort the data processing results by perplexity, score the data processing results from a fluency dimension, and obtain a fluency scoring result; The fourth evaluation submodule is used to score the data processing result from the dimension of repeatability using a preset repeatability evaluation rule to obtain a repeatability scoring result; A fifth evaluation submodule, configured to score the data processing result from a rhythm dimension using a preset rhyme judgment rule to obtain a rhythm scoring result; The sixth evaluation submodule is used to score the data processing result from the rhetorical skill dimension using a preset rhetorical judgment method to obtain a rhetorical scoring result; The evaluation result acquisition submodule is used to perform weighted calculation on the format scoring result, the theme expression scoring result, the fluency scoring result, the repetition scoring result, the rhythm scoring result, and the rhetoric scoring result according to a preset weight ratio to obtain the data processing result.

[0021] Optionally, the device further comprises: A first weight ratio adjustment module, used for adjusting the weight ratio according to the data processing result; A second weight ratio adjustment module, used to adjust the weight ratio according to user feedback information; The third weight ratio adjustment module is used to iteratively optimize different combinations of the weight ratios through a preset optimization algorithm to adjust the weight ratios.

[0022] A fourth aspect of an embodiment of the present application provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps in the method described in the first aspect of the present application are implemented.

[0023] A fifth aspect of an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method described in the first aspect of the present application are implemented.

[0024] Using the data processing method provided in the present application, task data corresponding to a data processing task is received; the data processing task is executed through at least one first model to obtain at least one data processing result; the at least one data processing result is input into a second model; the data processing result is evaluated from at least one preset dimension through the second model to obtain at least one evaluation result; and based on the evaluation result, a target data processing result is determined from the at least one data processing result.

[0025] In the present application, at least one first model is pre-loaded in the model evaluation system, and the task data is input into the first model to obtain the corresponding data processing result, and then the data processing result is evaluated in multiple dimensions through the second model to obtain the data processing result. This method can freely load the first model that needs to be evaluated, evaluate multiple first models at the same time, and evaluate the data processing results from multiple dimensions. Then, through the evaluation results, the performance differences of different models can be accurately grasped, and the optimal data processing result can be selected. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0027] Figure 1 It is a structural diagram of a model evaluation system proposed in one embodiment of the present application; Figure 2 is an architecture diagram of a model evaluation system proposed in one embodiment of the present application; Figure 3 is a workflow diagram of a model evaluation system proposed in one embodiment of the present application; Figure 4 is a system evaluation flow chart proposed in an embodiment of the present application; Figure 5 is a flow chart of a data processing method proposed in one embodiment of the present application; Figure 6 is a flow chart of model parameter setting proposed in one embodiment of the present application; Figure 7 is a schematic diagram of a data processing device proposed in an embodiment of the present application; Figure 8 It is a schematic diagram of an electronic device proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] refer to Figure 1 , Figure 1 is a schematic diagram of the structure of the model evaluation system proposed in the embodiment of the present application, such as Figure 1 As shown, the system includes: A data processing unit, used to call at least one first model to perform a data processing task and obtain at least one data processing result, wherein the at least one first model is a different model; a result evaluation unit, used to evaluate the at least one data processing result respectively through a second model to obtain at least one evaluation result, and output the evaluation result to a user page, wherein the evaluation result is used to compare the differences between the data processing results of different first models; a user feedback acquisition unit, used to obtain user feedback information on the evaluation result, so that the second model optimizes the evaluation logic according to the feedback information.

[0030] In this embodiment, the first model is an artificial intelligence model that performs data processing tasks. It can perform corresponding data processing tasks according to the task requirements proposed by the user. For example, if the user needs to generate a poem through the model, the data processing task to be performed is the poem generation task. At this time, the user inputs the corresponding task data, such as the main body of the poem and the format of the poem, and the first model can generate the corresponding poem. The system can use multiple different first models to perform data processing tasks at the same time, and then obtain multiple data processing results.

[0031] In this embodiment, the second model is a model for evaluating text content based on natural language processing technology (NLP, Natural Language Processing) and trained using Transformer (a deep learning framework). Taking poetry recognition as an example, when training the second model, the processed poems that meet the standard format are used as training materials and input into the second model to be trained. The parameters of the second model are adjusted to obtain a trained second model, so that it has the performance of evaluating and scoring the format, theme, rhythm and other elements of the poem, and then a comprehensive evaluation is made for a poem. This evaluation score can reflect the quality of the data processing results, and then reflect the performance of the first model. By comparing multiple evaluation scores, the differences between the data processing results of the first model can be determined.

[0032] The result evaluation unit comprises: The dimension evaluation subunit is used to evaluate the data processing result from at least one preset dimension to obtain the evaluation result.

[0033] In this embodiment, the second model evaluates the data processing results through at least one preset dimension, and a corresponding algorithm is set for the evaluation of each dimension. For example, when evaluating poetry, the content of poetry will be evaluated from multiple dimensions such as format, theme expression, fluency, repetition, rhythm, and rhetorical skills.

[0034] The user feedback acquisition unit includes at least one of the following: a first user feedback acquisition subunit, used to obtain the user's evaluation selection for each of the evaluation results, and the evaluation selection includes any one of the following: over-evaluation, normal evaluation, and under-evaluation; a second user feedback acquisition subunit, used to obtain the user's evaluation content for each of the evaluation results, and the evaluation content includes a score and evaluation information for each of the evaluation results.

[0035] In this embodiment, the user can evaluate each evaluation result in the result display page of the system. The options include over-evaluation, normal evaluation, and under-evaluation. Alternatively, the user can directly score the evaluation result and write down the corresponding evaluation information, including information such as the reason for the evaluation. The user feedback acquisition unit obtains the user's evaluation selection or evaluation information for optimizing the parameters of the second model.

[0036] The system further comprises: A model loading unit, used to load the corresponding first model according to the model information input by the user; A state setting unit is used to set the use state of the first model, and the use state includes an enabled state and an abandoned state; a model information display unit is used to display the model information of the first model; and a parameter configuration unit is used to configure the parameters of the first model according to the model configuration parameters.

[0037] In this embodiment, the user can load the first model in the system and write the model name, loading path and other information. The system can then load the corresponding first model according to the loading path. The user can also set the usage status of each model. The model information of each model will be displayed in the model list. The user can also configure the model parameters independently by entering the corresponding parameters for configuration.

[0038] The system also includes: a file receiving unit, used to receive a task file through a first interface, wherein the task file includes task data corresponding to the data processing task; a result file acquisition unit, used to output a result file corresponding to the data processing task through a second interface, wherein the result file includes the data processing results of the first model and the evaluation results of the second model.

[0039] In this embodiment, the system can receive a task file containing task data according to a preset first interface. After executing the data processing task, it can output a result file containing corresponding result data through a second interface. The user can provide a task file and receive a corresponding result file.

[0040] The system further comprises: An evaluation result display unit is used to display the evaluation score corresponding to each of the first models and the evaluation reason corresponding to the evaluation score; an image generation unit is used to calculate the average score of the evaluation scores corresponding to all the first models when all the first models have completed the data processing task, and generate a corresponding score distribution graph, and mark the average score in the score distribution graph; an evaluation history display unit is used to display the data processing result corresponding to each of the data processing tasks executed by the first model, and the evaluation record corresponding to the data processing result, wherein the evaluation record includes the task content of the data processing task, the evaluation result corresponding to the data processing result, and the evaluation reason.

[0041] In this embodiment, the evaluation results can be recorded and counted. The user can view the output results of each first model and second model recorded after the execution of each data processing task in the corresponding page, as well as detailed information on the evaluation results corresponding to each data processing task, including evaluation content, evaluation reasons, evaluation scores and other information. The average score of multiple evaluation results can also be calculated to generate a score distribution graph, and the average score can be marked in the score distribution graph, so that the user can intuitively see the score of the data processing results output by each model.

[0042] refer to Figure 2 , Figure 2 is an architecture diagram of a model evaluation system proposed in an embodiment of the present application, such as Figure 2 As shown, the front end of the model evaluation system includes a home page, a model management page, and an evaluation history page. The home page includes functions for displaying model functions, model reasoning functions, model evaluation functions, and visually displaying model evaluation scores. The model management page includes functions for model download, model configuration, model parameter setting, and model switch. The evaluation history page includes the number of copies of each evaluation of each first model and related data. The details page is the detailed data of the first model at each reasoning. The reasoning evaluation interface in the program interface can import user-defined content that needs to be evaluated, and the export interface outputs the corresponding evaluation score. The system relies on multiple first models for reasoning, relies on the second model for scoring, and relies on the database to provide relevant function implementations.

[0043] refer to Figure 3 , Figure 3 is a workflow diagram of a model evaluation system proposed in an embodiment of the present application, such as Figure 3 As shown, when users use the system, the home page will be opened by default. The home page mainly performs the settings and displays of the inference content and the second model evaluation content. Users can switch to the model setting page to perform model-related operations, such as model download, parameter setting, enabling / disabling, etc. Users can view the evaluated content and corresponding details in the historical evaluation page. When the user enters the home page, the system displays the models started by the current user settings. The system will perform inference by calling these started models. At the same time, the user can choose automatic evaluation, manual evaluation, and data processing in the form of self-evaluation. In the case of automatic evaluation, the system will automatically evaluate the data processing results through the second model after all the first models have completed inference. In the case of manual evaluation, the user clicks the evaluation button on the home page to evaluate. If it is a self-evaluation, the user will fill in the evaluation information, i.e., the score, below each model after all the model inferences are completed, and then submit it to the second model for reference by the large model.

[0044] After entering the management page, the management page displays a model list, including downloaded models, local models, online models, etc. The online model will display status information such as waiting to be downloaded, downloading, and downloading completed, as well as download path and parameter configuration information. The local model list displays information related to the local model, such as local path, online path, parameter configuration, etc. The online model will display information such as online address and parameter configuration. Both the downloaded model and the local model need to configure model parameters.

[0045] After entering the historical evaluation page, the system will display the evaluation record of each first model in the evaluation list in chronological order, which includes the model information of the models that have been evaluated. You can click Details to enter the evaluation details page, which includes reasoning content, data processing results, user self-evaluation information, etc.

[0046] The system also supports user feedback, and users can rate the satisfaction of the evaluation score of the second model. The second model optimizes its own evaluation logic based on user feedback to make it more in line with user requirements. The second model determines whether the current evaluation score of the model performance is high, low or normal based on the user's rating, and the second model can optimize and adjust its own parameters based on the evaluation score fed back by the user, so that the evaluation score of the second model better meets the user's requirements.

[0047] refer to Figure 3 , Figure 3 is a system evaluation flow chart proposed in an embodiment of the present application, such as Figure 3 As shown, when the user operates the system, first add the model to be used on the model configuration page, enter the name of the configuration model, local path (required for local model calls), API (Application Programming Interface) address (required for online model calls), model parameters and corresponding values, and save them. At this time, the system will enable the model and make interface calls with the configured address and parameters.

[0048] Then select the model you want to use, enable or disable the model on the model configuration page. After the configuration is complete, click the Start Inference button to perform inference and display the data processing results on the home page.

[0049] The homepage is the main page for users to conduct model inference evaluation. It mainly includes the user input inference content control, multi-model data processing result display control, large model evaluation method control, large model evaluation scoring result control, large model evaluation reason control, and user self-evaluation control. The user enters the content to be inferred in the inference content control and clicks the start inference control, and the system starts inference. Multiple models are inferred synchronously, and the results will be displayed in real time in the multi-model data processing result display control until all models are inferred.

[0050] After all the first models are completely inferred, the system automatically evaluates the data processing results according to the instructions received by the large model evaluation mode control, or waits for the user to click the manual evaluation button for evaluation. After the evaluation is completed, the user can click the re-evaluation button to evaluate the data processing results again. The autonomous evaluation control in the system can receive the custom content sent by the user, evaluate the custom content, and obtain the corresponding evaluation score.

[0051] After the evaluation is completed, the evaluation score and the corresponding evaluation reason will be displayed below the data processing results of each model in the visualization interface. Finally, an evaluation average score and score distribution chart are generated to intuitively display the score. Users can go to the evaluation history page to view all evaluation records. The evaluation records will display information such as reasoning content, reasoning average score, and score of each model. Users can call the local interface to automatically reason and evaluate local files. The local file content contains all the content that the user wants to reason about. Then, through the interface, the final multi-model data processing results and evaluation scores are obtained.

[0052] refer to Figure 4 , Figure 4 1 is a flow chart of a data processing method proposed in an embodiment of the present application, which is applied to a model evaluation system. Figure 4 As shown, the method comprises the following steps: S11: Receive task data corresponding to the data processing task.

[0053] In this embodiment, at least one first model is preloaded in the model evaluation system, and these first models can be obtained locally or downloaded from a model library on the network through a program interface. The task data corresponding to the data processing task is received through the at least one preloaded first model.

[0054] For example, the first model is a language model that can generate poetry, and the task data is "generate a five-character quatrain about the moon."

[0055] S12: Execute the data processing task through at least one first model to obtain at least one data processing result.

[0056] In this embodiment, the data processing result is the output obtained by the first model according to the input task data.

[0057] In this embodiment, multiple first models receive task data at the same time and generate corresponding data processing results according to the task data. The first models are all trained models, and process the input task data according to the parameters learned during training to obtain corresponding data processing results.

[0058] For example, the first model generates a five-character quatrain containing the moon or the factor of the moon according to the input task data "generate a five-character quatrain about the moon".

[0059] S13: Inputting the at least one data processing result into the second model.

[0060] In this embodiment, when multiple data processing results are obtained by the first model, the multiple data processing results are respectively input into the second model.

[0061] S14: Evaluate the data processing result from at least one preset dimension using the second model to obtain at least one evaluation result.

[0062] In this embodiment, the evaluation result is an evaluation score obtained after evaluating the data processing result of the first model.

[0063] In this embodiment, a multi-dimensional evaluation is performed on the input data processing result through the second model to obtain an evaluation result. Multiple aspects need to be taken into consideration during the evaluation, and then multiple dimensions are set to evaluate the data processing result.

[0064] In this embodiment, taking poetry as an example, the second model is mainly based on natural language processing technology (NLP, Natural Language Processing), and uses Transformer (a deep learning framework) technology to train the model, and train the model to be a model that can understand the language structure and style of poetry.

[0065] In this embodiment, taking poetry evaluation as an example, most current poetry evaluation methods use certain rules and BLEU (bilingual evaluation under study) to evaluate. When using rule evaluation alone, only the format can be evaluated, and other evaluation effects cannot be achieved. Although the BLEU method has fast calculation speed, does not distinguish between languages, and has better applicability, it only considers the similarity of N-gram grammars and does not have semantic level evaluation.

[0066] In this embodiment, the content of the poem is evaluated from multiple dimensions, including format, theme expression, fluency, repetition, rhythm, rhetorical skills, etc.

[0067] In this embodiment, the data processing result is scored by at least one preset evaluation dimension, and the specific steps of obtaining the data processing result include: S14-1: Use preset format matching rules to score the data processing result from a format dimension to obtain a format scoring result.

[0068] In this embodiment, different inference contents have different format requirements. You can set the matching rules according to the actual situation. Use the preset format matching rules to score the data processing results from the format dimension to obtain the format scoring results. When building the second model, add the set format matching rules to the model. During the training process, the second model will adjust the corresponding parameters of the format matching rules, thereby continuously improving the format matching rules.

[0069] In this implementation, taking poetry as an example, the second model supports the recognition of formats including five-character quatrains, seven-character quatrains, five-character regulated verses, and seven-character regulated verses. It also supports the ci tunes such as Partridge Sky, Huanxisha, Manjianghong, Bodhisattva Man, Dielianhua, Linjiangxian, Shuidiaogetou, Qingpingle, and Niannujiao, and supports different word counts for the same ci tune, which can better perform format verification and evaluation on different types of poetry. The model uses rule matching. The matching rule is to segment the content according to punctuation marks, and judge whether each paragraph of the segmented content in Nie Rong meets the corresponding format word count. If it meets the requirements, the data processing result is 100 points, and if it does not meet the format, it is 0 points. For example, each sentence of a five-character quatrain has five words. If each sentence in the data processing result has five words, it is judged to be full marks.

[0070] The corresponding evaluation formula is: (1) Among them, format_score represents the format score, f(x) represents the final score, and weight represents the weight. From the formula, we can see that the score result is divided into two parts. If format_score is 0, the overall score is 0. If format_score is not 0, the final score is the score of each dimension multiplied by the score of each dimension, and then summed to get the final result.

[0071] S14-2: By calculating the consistency score between the content of the data processing result and the theme, the data processing result is scored from the theme expression dimension to obtain a theme expression scoring result.

[0072] In this embodiment, the consistency score between the content and the topic of the data processing result is also called the answer fidelity score, which is an evaluation indicator obtained using LLM (Large Language Model) and is used to evaluate whether the topic of a content meets the proposed requirements. The second model integrates the language recognition function of the large language model, and continuously adjusts the parameters for content recognition during the training process to accurately obtain the answer fidelity score of each inference content.

[0073] In this embodiment, the answer fidelity score of the data processing result is calculated, and the data processing result is scored from the dimension of topic expression to obtain the corresponding topic expression scoring result.

[0074] In this embodiment, the second model is used to decompose the poem into multiple statements, and the consistency of each statement with the theme is checked. A "fidelity score" is calculated based on the ratio of the number of supported statements to the total number of statements. The score is added in the form of 5 points for each statement that meets the criteria, with a maximum of 100 points.

[0075] (2) Where score is the final score, w is the weight, k is the number to be added, and y is the score to be added when the result matches, where y is always equal to 5. The weighted sum is compared with 100 and the smallest value is taken. Finally, the weight w is multiplied to get the final score.

[0076] In this embodiment, the second model fine-tunes the model to make a deeper fit to the theme. By using the model fine-tuning method, the poem content is compared with the user questions and answers to evaluate the fit between the content and the theme, which includes the theme words and words related to the theme words. For example, "write a five-character quatrain with Mid-Autumn Festival as the theme", then the theme word is "Mid-Autumn Festival", but if the title contains "moon", "Chang'e", etc., it will also be evaluated as being in line with the theme.

[0077] S14-3: sorting the data processing results by perplexity, scoring the data processing results from a fluency dimension, and obtaining a fluency scoring result.

[0078] In this embodiment, the basic idea of ​​perplexity ranking (PPL) is to calculate the probability of a sentence appearing. The larger the score, the lower the probability of appearing. That is, the larger the score, the lower the probability of appearing sequence, and the more difficult it is to have a smooth text. The lower the score, the easier it is to have a smooth text. Then, based on this idea, the poem is calculated to determine the fluency of the poem.

[0079] In this embodiment, the probability of a sentence appearing is calculated and raised to the power of 1 / N. The formula is as follows: (3) Among them, w1 w2…wN represent N tokens in a sentence (a token corresponds to the dictionary in the language model, and for Chinese, it may be a character or a word), and P is the probability of generating each word.

[0080] From the above formula, we can see that when using PPL sorting to analyze a poem, the lower the score, the more likely it is that the poem is a fluent text and the higher the fluency.

[0081] The scoring formula is: (4) Where score is the final score, w is the weight, and PPL is the result of the above calculation. Finally, multiply it by the weight w to get the final score.

[0082] In this embodiment, the fluency is mainly in the form of a formula, and the final results are sorted by perplexity. The fluency of the data processing results is judged by the formula evaluation standard, which greatly improves the efficiency of the evaluation.

[0083] S14-4: Using preset duplication evaluation rules, score the data processing results from the duplication dimension to obtain a duplication scoring result.

[0084] In this embodiment, the preset repetition evaluation rules are used to score the data processing results from the repetition dimension to obtain the repetition scoring results. The repetition evaluation rules are added when the second model is constructed. When the model is trained, the relevant parameters of the repetition evaluation rules can be iteratively adjusted, so as to better process and analyze the poems through the rules.

[0085] In this embodiment, the repetition degree is evaluated by rules, and the poem content is split word by word, and then the repetition degree is determined. The higher the repetition degree, the lower the score. This evaluation dimension is in the form of deduction, with a starting score of 100 points, 5 points deducted for each repetition, the highest score is 100 points, and the lowest score is 0 points. The formula is: (5) Where score is the final score, w is the weight, k is the number to be added, and y is the score of the number of repeated words, where y=5 is a constant value. 100 points minus the score that matches is compared with 0, and the larger one is taken as the final result, which is multiplied by the weight w to get the final score.

[0086] In this embodiment, the repetition degree is the measurement of the repetition rate of the poem content in the final result. For a poem, if there is too much repeated content, it may lead to a rather rigid result. Of course, there are some good poems that use repeated content, such as "Jiji Fujiji". Therefore, the method of subtracting the repeated number from the total is adopted here. The repeated content is subtracted to obtain the final result, which not only judges the repeated number but also optimizes the result in a form from high to low, avoiding the situation of overly weakening repeated words.

[0087] S14-5: Use the pre-set rhyme judgment rule to score the data processing result from the prosody dimension to obtain the prosody score result.

[0088] In this embodiment, use the pre-set rhyme judgment rule to score the data processing result from the prosody dimension to obtain the prosody score result. When constructing the second model, the rhyme judgment rule is added, and the relevant parameters can be continuously optimized with the training of the model, so as to more accurately perceive and evaluate the prosody and rhythm of a text content.

[0089] In this embodiment, the method of converting Chinese characters to pinyin is adopted. The last character of each sentence is converted into pinyin, and it is judged whether it meets the rhyme. According to the degree of satisfaction, a score is given. The higher the proportion of rhyming sentences that are satisfied, the higher the score. The formula is as follows: (6) Where score is the final score, w represents the weight, n represents the total number of rhymes that meet the requirements, t is the maximum number of rhymes for the current type of poem, multiplying by 100 is to obtain a value within 100 after calculating the proportion, which is convenient to keep the same unit as other dimensions for statistics. Multiply by the weight w to get the final score.

[0090] In this embodiment, the second model adopts the method of pinyin recognition, compares the pinyin of the last character, and judges whether it meets the rhyme, which can better evaluate the prosody.

[0091] S14-6: Use the pre-set rhetoric judgment method to score the data processing result from the rhetoric skill dimension to obtain the rhetoric score result.

[0092] In this embodiment, when generating an article or a poem, it is necessary to judge its rhetorical devices. Use the pre-set rhetoric judgment method to score the data processing result from the rhetoric skill dimension to obtain the rhetoric score result. During the training process of the second model, the parameters corresponding to the rhetoric judgment method will be continuously adjusted, so that the model can more accurately judge the rhetorical methods used in the text content.

[0093] In this embodiment, the second model translates the input poem into modern text, and then judges whether the modern text contains metaphors, personification and other rhetorical devices, and then scores the poem. The more rhetorical devices there are, the higher the score. If there is one rhetorical device, 5 points will be added, up to 100 points. The formula is as follows: (7) Where score is the final score, w is the weight, k is the number to be added, and y is the score added when there is rhetoric, where y = 5. The result is multiplied by the weight w to get the final score.

[0094] In this embodiment, the second model uses the method of translating ancient poetry into modern texts to determine the rhetorical techniques. This method not only simplifies the judgment logic, but also increases the accuracy of the judgment. The large model is trained to judge different rhetorical techniques through fine-tuning. It can judge metaphors, parallelisms, personification and other techniques. By judging and evaluating different techniques, the dimension of poetry evaluation is increased.

[0095] S14-7: Perform weighted calculation on the format scoring result, the theme expression scoring result, the fluency scoring result, the repetition scoring result, the rhythm scoring result, and the rhetoric scoring result according to a preset weight ratio to obtain the data processing result.

[0096] In this embodiment, the format scoring result, the theme expression scoring result, the fluency scoring result, the repetition scoring result, the rhythm scoring result, and the rhetoric scoring result are weighted and calculated according to a preset weight ratio to obtain a data processing result. The data processing result is a comprehensive score.

[0097] In this embodiment, the second model supports specifying the weight of each evaluation dimension in the form of parameters. For example, if the theme expression is the most important, it can be set to the highest weight, with 100 as the highest score and 0 as the lowest score. At present, the sum of all parameter weights is 100. The system will first select the dimension with the largest weight, and then select the dimension with the second largest weight. If the sum of the previous parameter settings reaches 100, the subsequent parameters will be automatically set to 0. If there are dimensions with the same weight, they will be selected according to the system preset order, such as fluency and repetition. The system sets fluency priority over repetition priority. If the two weight settings are consistent, the system will give priority to fluency for weight addition. If there is any remaining weight, it will be allocated to repetition.

[0098] In this embodiment, the initial weight ratio is based on the expert's experience and opinions as the starting weights. Initially, the expert's opinions are theme expression: 40%, fluency: 25%, repetition: 15%, rhythm: 10%, and rhetoric skills: 10%.

[0099] In this embodiment, the method further includes: S14-8: Adjust the weight ratio according to the data processing result.

[0100] In this embodiment, a feedback loop is established. After obtaining the data processing results, the model performance model adjusts the proportion of the scoring criteria by automatically feeding back the evaluation score. The feedback loop can be implemented through methods such as reinforcement learning or genetic algorithms. Through these algorithms, the model gradually optimizes internal parameters according to the evaluation score.

[0101] S14-9: Adjust the weight ratio according to user feedback information.

[0102] In this embodiment, the user feedback information refers to the user's autonomous rating of the data processing results of each first model, and also includes the user's rating of the second model.

[0103] In this embodiment, user feedback information is introduced to further adjust the proportion of the scoring criteria. The user can score the data processing results or provide other forms of feedback, and the parameter optimization of the second model is guided by the user feedback information.

[0104] S14-10: Iteratively optimize different combinations of the weight ratios through a preset optimization algorithm to adjust the weight ratios.

[0105] In this embodiment, hyperparameter optimization technology can be used to automatically adjust the weight ratio to maximize the overall score as the optimization goal to adjust the weight ratio. For example, a Bayesian model is used for optimization, and a corresponding acquisition function is designed to iteratively optimize the model. Different ratio combinations are automatically tried in a given search space to find the optimal solution.

[0106] In this embodiment, the expert opinion rating ratio is used as the initial weight ratio of each dimension, and then a set of optimized weight ratios is found using hyperparameter optimization techniques (such as Bayesian optimization), and then the optimized weight ratios are input into the second model, and these ratios are further fine-tuned through feedback loops (such as reinforcement learning). The feedback loop can automatically adjust the ratios according to the performance of the poems generated by the model in real or simulated environments to achieve better results.

[0107] In addition to the automatic feedback loop, user feedback is introduced to guide the optimization process. Users can rate the poems generated by the model or provide other forms of feedback. These user feedbacks serve as additional input data to adjust the optimization objectives or weights in the feedback loop. For example, if most users think that the rhetoric part of the poem is not good enough, the system can automatically increase the weight of the rhetoric score and adjust the weights of other parts accordingly.

[0108] For example, after continuous optimization, the weight ratios obtained are: theme expression: 38.3%, fluency: 30.5%, repetition: 10.6%, rhythm: 11.2%, and rhetorical skills: 9.4%.

[0109] S15: Determine a target data processing result from the at least one data processing result according to the evaluation result.

[0110] In this embodiment, after obtaining the evaluation result corresponding to each data processing result, the evaluation result with the highest score is determined from the multiple data processing results and is used as the target data processing result.

[0111] For example, when the data processing task is to generate a poem, the poem with the highest score is selected as the poem, and the first model that generates the poem is determined to be the optimal model.

[0112] In this embodiment, a comprehensive optimization framework is constructed, which can integrate multiple optimization strategies such as hyperparameter optimization, feedback loop and user feedback. In this framework, different optimization stages and priorities are set. Through multiple strategies, the weight of each dimension of the evaluation text of the second model is continuously optimized, so that the second model makes a more accurate evaluation of the data processing results, effectively improving the efficiency and accuracy of the second model evaluation.

[0113] In another embodiment of the present application, before receiving the task data through at least one pre-loaded first model, the method further includes: S21: Receive model information of the first model to be loaded.

[0114] In this embodiment, the model information includes the model name, local path, program interface address, model parameters, and model parameter values.

[0115] In this embodiment, the system first receives model information of at least one first model to be loaded.

[0116] In this embodiment, the model name, model address, model parameter type, and model parameter value included in the model information are read.

[0117] S22: When the model address included in the model information is a local path, obtain the first model from a storage location corresponding to the local path.

[0118] In this embodiment, when the model address is a local path, it means that the model is stored locally, so the first model can be obtained from the storage location corresponding to the local path.

[0119] S23: When the model address included in the model information is a program interface address, download the first model through the program interface address.

[0120] In this embodiment, when the model address is a program interface address, the first model needs to be downloaded from the program interface address.

[0121] S24: According to the model parameter type included in the model information, the corresponding model parameter value is input into the first model.

[0122] In this embodiment, the model parameter type specifies the specific type of the model parameter, and the model parameter value is the specific value of the parameter.

[0123] In this embodiment, the model information includes the type of model parameters and the corresponding model parameter values. According to the type of model parameters, the corresponding model parameter values ​​are loaded into the first model, and the first model is loaded into the model evaluation system.

[0124] S25: Setting the first model to an enabled state.

[0125] In this embodiment, when the first model is loaded, the first model is set to a startup state.

[0126] In this embodiment, it is also necessary to set the parameters of the model, refer to Figure 5 , Figure 5 is a flow chart of model parameter setting proposed in an embodiment of the present application, such as Figure 5 As shown, the user selects the model that needs to set parameters on the page, fills in the request header information, fills in the specific parameters of the request, fills in the URL (network address) request parameters, saves the filled information, and the system verifies the parameters. If the parameter verification is correct, the filled parameters are stored in the database. When the model is used, the stored parameters are called to load them into the model. For model-related parameters, the following parameter data table is designed. The fields of the main parameters are shown in Tables 1 and 2. Table 1 is the model parameter type table, and Table 2 is the model parameter value table:

[0127] Table 1

[0128] Table 2 In this embodiment, the user can load the corresponding model into the system according to the needs, and set the corresponding parameters according to the needs, so as to achieve flexible expansion of the first model and meet the needs of the user.

[0129] In another embodiment of the present application, after obtaining the performance evaluation score, the average score of all the data processing results is calculated.

[0130] For example, there are three data processing results, 95, 96, and 97, and the average score is 96.

[0131] In this embodiment, after calculating the average score, a score distribution diagram of the data processing results is generated, and the average score of the performance evaluation scores is marked in the score distribution diagram, so that it can be intuitively seen whether the performance of the first model is higher than the average.

[0132] In this embodiment, by generating an average score and a score distribution diagram, the score of each model can be intuitively seen, which is helpful for the user to grasp the specific performance of each first model.

[0133] In another embodiment of the present application, the method further includes: S31: storing the data processing result corresponding to the first model, the data processing result and the corresponding evaluation reason in the evaluation record corresponding to the first model.

[0134] In this embodiment, for each first model, the data processing result of the model, the data processing result and the corresponding evaluation reason are stored in the evaluation record corresponding to the first model, so that the user can view it at any time.

[0135] S32: When receiving a request for viewing the evaluation record corresponding to the first model, display the evaluation record.

[0136] In this embodiment, when the system receives a request to view the evaluation record of the first model, the evaluation record is displayed.

[0137] In this embodiment, the user can view the evaluation records at any time to determine the overall performance of each first model and have a global understanding of the performance of the model.

[0138] In another embodiment of the present application, the method further includes: S41: When a re-evaluation instruction is received, re-evaluate the at least one data processing result using the second model.

[0139] In this embodiment, when the system receives a re-evaluation instruction, the model evaluation system re-evaluates the data processing results through the second model. Because the parameters can be adjusted through a feedback loop during the evaluation process, the score may be different each time, and multiple evaluations have greater reference significance.

[0140] In this embodiment, the user can issue a re-evaluation instruction at any time to re-evaluate the evaluation score. The user can grasp the performance status of each first model as a whole based on the results of multiple evaluations.

[0141] In another embodiment of the present application, the system counts the evaluation scores of the second model. If the score of a first model is lower than a preset score threshold in multiple evaluations, the first model is marked as a model that is not recommended for use, to remind the user that the performance of the first model is poor and cannot provide good services to the user. Alternatively, the scores of each first model are sorted. If a first model is ranked low multiple times, the first model is marked as a not recommended model. In this way, users can be helped to filter out high-performance first models and eliminate low-performance first models, so that users can quickly find high-performance models and improve the user experience.

[0142] Based on the same inventive concept, an embodiment of the present application provides a data processing device, which is applied to a model evaluation system. Figure 7 , Figure 7 FIG. 7 is a schematic diagram of a data processing device 700 proposed in an embodiment of the present application. Figure 7 As shown, the device comprises: The task data receiving module 701 is used to receive the task data corresponding to the data processing task; A data processing result acquisition module 702 is used to execute a data processing task through the at least one first model to obtain at least one data processing result; A data processing result input module 703, used to input the at least one data processing result into the second model; A data processing result evaluation module 704 is used to evaluate the data processing result from at least one preset dimension using the second model to obtain at least one evaluation result; The data processing result determination module 705 is used to determine a target data processing result from the at least one data processing result according to the evaluation result.

[0143] Optionally, the device further comprises: A model information receiving module, used to receive model information of at least one first model to be loaded; A first model acquisition submodule, configured to acquire the first model from a storage location corresponding to the local path when the model address is a local path; A second model acquisition submodule, configured to download the first model through the program interface address when the model address is a program interface address; A parameter input submodule, used for inputting the corresponding model parameter value into the first model according to the model parameter type; The model activation module is used to set the first model to an activated state when the first model is loaded.

[0144] Optionally, the device further comprises: An evaluation record module, used for storing the data processing result corresponding to the first model, the data processing result and the corresponding evaluation reason in an evaluation record corresponding to the first model; The evaluation record display module is used to display the evaluation record when receiving a request to view the evaluation record corresponding to the first model.

[0145] Optionally, the device further comprises: A re-evaluation module is used to re-evaluate the at least one data processing result by using the second model when a re-evaluation instruction is received.

[0146] Optionally, the data processing result evaluation module includes: A first evaluation submodule is used to score the data processing result from the format dimension using a preset format matching rule to obtain a format scoring result; The second evaluation submodule is used to score the data processing result from the dimension of theme expression by calculating the consistency score between the content of the data processing result and the theme, so as to obtain a theme expression scoring result; A third evaluation submodule is used to sort the data processing results by perplexity, score the data processing results from a fluency dimension, and obtain a fluency scoring result; The fourth evaluation submodule is used to score the data processing result from the dimension of repeatability using a preset repeatability evaluation rule to obtain a repeatability scoring result; A fifth evaluation submodule, configured to score the data processing result from a rhythm dimension using a preset rhyme judgment rule to obtain a rhythm scoring result; The sixth evaluation submodule is used to score the data processing result from the rhetorical skill dimension using a preset rhetorical judgment method to obtain a rhetorical scoring result; The evaluation result acquisition submodule is used to perform weighted calculation on the format scoring result, the theme expression scoring result, the fluency scoring result, the repetition scoring result, the rhythm scoring result, and the rhetoric scoring result according to a preset weight ratio to obtain the data processing result.

[0147] Optionally, the device further comprises: A first weight ratio adjustment module, used for adjusting the weight ratio according to the data processing result; A second weight ratio adjustment module, used to adjust the weight ratio according to user feedback information; The third weight ratio adjustment module is used to iteratively optimize different combinations of the weight ratios through a preset optimization algorithm to adjust the weight ratios.

[0148] Based on the same inventive concept, another embodiment of the present application provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the data processing method described in any of the above embodiments of the present application.

[0149] Based on the same inventive concept, another embodiment of the present application provides an electronic device, Figure 8 It is a schematic diagram of an electronic device 800 proposed in one embodiment of the present application, including a memory 802, a processor 801 and a computer program stored in the memory and executable on the processor, and the processor, when executed, implements the steps in the data processing method described in any of the above embodiments of the present application.

[0150] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0151] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0152] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, devices, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0153] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0154] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0155] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0156] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0157] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0158] The above is a detailed introduction to a model evaluation system, data processing method, device, equipment and medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A model evaluation system, characterized in that: The system comprises: A data processing unit, configured to call at least one first model to perform a data processing task and obtain at least one data processing result, wherein the at least one first model is a different model; A result evaluation unit, used to evaluate the at least one data processing result respectively through the second model to obtain at least one evaluation result, and output the evaluation result to a user page, wherein the evaluation result is used to compare the differences between the data processing results of different first models; A user feedback acquisition unit is used to obtain user feedback information on the evaluation result, so that the second model optimizes the evaluation logic according to the feedback information.

2. The model evaluation system according to claim 1, characterized in that: The result evaluation unit comprises: The dimension evaluation subunit is used to evaluate the data processing result from at least one preset dimension to obtain the evaluation result.

3. The model evaluation system according to claim 1, characterized in that: The user feedback acquisition unit includes at least one of the following: A first user feedback acquisition subunit is used to acquire the evaluation selection made by the user on each of the evaluation results, wherein the evaluation selection includes any one of the following: over-evaluation, normal evaluation, and under-evaluation; The second user feedback acquisition subunit is used to acquire the user's evaluation content for each of the evaluation results, where the evaluation content includes a score and evaluation information for each of the evaluation results.

4. The model evaluation system according to claim 1, characterized in that: The system further comprises: A model loading unit, used to load the corresponding first model according to the model information input by the user; A state setting unit, used to set the use state of the first model, wherein the use state includes an enabled state and a disabled state; A model information display unit, used to display model information of the first model; A parameter configuration unit is used to configure parameters of the first model according to the model configuration parameters.

5. The model evaluation system according to claim 1, characterized in that: The system further comprises: A file receiving unit, configured to receive a task file through a first interface, wherein the task file includes task data corresponding to the data processing task; A result file acquisition unit is used to output a result file corresponding to the data processing task through a second interface, wherein the result file includes the data processing result of the first model and the evaluation result of the second model.

6. The model evaluation system according to claim 1, characterized in that: The system further comprises: An evaluation result display unit, used to display the evaluation score corresponding to each of the first models and the evaluation reason corresponding to the evaluation score; an image generating unit, configured to calculate an average score of the evaluation scores corresponding to all the first models when all the first models have completed the data processing task, generate a corresponding score distribution graph, and mark the average score in the score distribution graph; An evaluation history display unit is used to display the data processing results corresponding to each of the data processing tasks executed by the first model, and the evaluation records corresponding to the data processing results, wherein the evaluation records include the task content of the data processing task, the evaluation results corresponding to the data processing results, and the reasons for the evaluation.

7. A data processing method, characterized in that: The method is based on the model evaluation system according to any one of claims 1 to 6, comprising: Receive task data corresponding to the data processing task; Execute the data processing task by at least one first model to obtain at least one data processing result; inputting the at least one data processing result into a second model; Evaluate the data processing result from at least one preset dimension using the second model to obtain at least one evaluation result; According to the evaluation result, a target data processing result is determined from the at least one data processing result.

8. The data processing method according to claim 7, characterized in that: Before performing the data processing task by at least one first model to obtain at least one data processing result, the method further includes: receiving model information of the first model to be loaded; In a case where the model address included in the model information is a local path, acquiring the first model from a storage location corresponding to the local path; In a case where the model address included in the model information is a program interface address, downloading the first model through the program interface address; According to the model parameter type included in the model information, inputting the corresponding model parameter value into the first model; Set the first model to an enabled state.

9. The data processing method according to claim 7, characterized in that: The method further comprises: storing the data processing result corresponding to the first model, the data processing result and the corresponding evaluation reason in the evaluation record corresponding to the first model; When a request for viewing the evaluation record corresponding to the first model is received, the evaluation record is displayed.

10. The data processing method according to claim 7, characterized in that: The method further comprises: When a re-evaluation instruction is received, the at least one data processing result is re-evaluated by using the second model.

11. The data processing method according to claim 7, characterized in that: The step of evaluating the data processing result from at least one preset dimension by using the second model to obtain at least one evaluation result includes: Using a preset format matching rule, scoring the data processing result from a format dimension to obtain a format scoring result; By calculating the consistency score between the content of the data processing result and the theme, the data processing result is scored from the theme expression dimension to obtain a theme expression scoring result; Sorting the data processing results according to the degree of perplexity, and scoring the data processing results from the dimension of fluency to obtain a fluency scoring result; Using a preset repetition evaluation rule, the data processing result is scored from the repetition dimension to obtain a repetition scoring result; Using a preset rhyme judgment rule, scoring the data processing result from a rhythm dimension to obtain a rhythm scoring result; Using a preset rhetoric judgment method, scoring the data processing results from the rhetoric skill dimension to obtain a rhetoric scoring result; The format scoring result, the theme expression scoring result, the fluency scoring result, the repetition scoring result, the rhythm scoring result, and the rhetoric scoring result are weighted calculated according to a preset weight ratio to obtain the data processing result.

12. The data processing method according to claim 11, characterized in that: The method further comprises: According to the evaluation result, the weight ratio is adjusted; Adjust the weight ratio according to user feedback information; By using a preset optimization algorithm, different combinations of the weight ratios are iteratively optimized to adjust the weight ratios.

13. A data processing device, characterized in that: The device comprises: A task data receiving module, used for receiving task data corresponding to a data processing task; A data processing result acquisition module, used to execute a data processing task through the at least one first model to obtain at least one data processing result; A data processing result input module, used for inputting the at least one data processing result into the second model; a data processing result evaluation module, configured to evaluate the data processing result from at least one preset dimension using the second model to obtain at least one evaluation result; A data processing result determination module is used to determine a target data processing result from the at least one data processing result according to the evaluation result.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 7 to 12 are implemented.

15. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 7 to 12 are implemented.

Images