Text processing method based on multi-dimensional evaluation, computer equipment and storage medium

Abstract

The invention provides a multi-dimensional evaluation text processing method, computer equipment and a storage medium, and the method comprises the steps: executing a text generation task through N target models, and carrying out the quality evaluation of a generated text output by each target model in other N-1 target models, so as to achieve the cross evaluation of the N target models on the generated texts, and improve the quality of the generated texts. And according to a cross evaluation result, determining an optimal text from the N generated texts as a generation result of the text generation task. According to the method, the corresponding evaluation indexes are adopted for evaluation for different text generation tasks, and the text reliability can be improved. Moreover, the optimal text is determined through cross evaluation for the generated texts of the N target models, and through cross evaluation, the advantages of different target models can be fully utilized, the blind area of a single model is made up, and the reliability of quality evaluation is further improved. By adopting the method, the text generation quality and the text processing reliability can be improved.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence technology and text processing technology, and particularly to a text processing method, computer device, and storage medium based on multi-dimensional evaluation. Background Art

[0002] With the rapid development of artificial intelligence technology, large language models (LLMs) have made remarkable progress in the field of text generation and are widely used in multiple scenarios such as content creation, intelligent customer service, assisted writing, automatic report generation, educational assistance, and medical record collation. These models are trained with massive amounts of text data, can understand context, and generate coherent text, greatly improving the efficiency of information processing and content creation, pushing the text generation ability to a new height, and making the generated content increasingly close to the human level in terms of fluency, relevance, and creativity. However, despite these remarkable achievements, the existing technology still faces some deficiencies in practical applications, restricting the further improvement of text generation quality and efficiency.

[0003] For example, there is a lack of an evaluation mechanism for the output results of large language models, resulting in the lack of reliability of the generated text in some cases. Summary of the Invention

[0004] To solve the existing technical problems, this application provides a text processing method, computer device, and storage medium based on multi-dimensional evaluation that can improve the reliability of the generated text.

[0005] In a first aspect, a text processing method based on multi-dimensional evaluation is provided. The method includes:

[0006] In response to a text generation task, determine evaluation indicators and weights for evaluating the text generation quality of the text generation task;

[0007] Input the text generation requirements of the text generation task into N target models respectively to obtain N generated texts output by the N target models; the target models are large language models;

[0008] For each generated text of each target model, generate evaluation prompt words corresponding to the N generated texts respectively according to the generated text, as well as each evaluation indicator and weight;

[0009] Input the evaluation prompt words of each generated text into the other N - 1 target models among the N target models respectively to obtain cross-evaluation results of the other N - 1 target models on the generated text;

[0010] According to the cross-evaluation results, determine the optimal text from the N generated texts as the generation result of the text generation task.

[0011] In a second aspect, a computer device is provided, including a processor and a memory connected to the processor. A computer program executable by the processor is stored on the memory. When the computer program is executed by the processor, the steps of the text processing method based on multi-dimensional evaluation described in the above embodiments are implemented.

[0012] In a third aspect, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps of the text processing method based on multi-dimensional evaluation described in the above embodiments are implemented.

[0013] In a fourth aspect, a computer-readable storage medium is provided. A computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the steps of the text processing method based on multi-dimensional evaluation described in the above embodiments are implemented.

[0014] For the text processing method based on multi-dimensional evaluation provided in the above embodiments, by using N target models to execute a text generation task, and for the generated text output by each target model, quality evaluation is performed on the other N - 1 target models, so as to implement cross-evaluation of the generated text by the N target models, and according to the cross-evaluation results, the optimal text is determined from the N generated texts as the generation result of the text generation task. This method determines the evaluation indicators and weights for evaluating the text generation quality of the text generation task, so as to use the corresponding evaluation indicators for evaluation for different text generation tasks, which is beneficial to improving the reliability of the text. Moreover, for the generated texts of the N target models, the optimal text is determined through cross-evaluation. Through cross-evaluation, the advantages of different target models can be fully utilized to make up for the blind spots of a single model, and further improve the reliability of quality evaluation. By adopting this method, the text generation quality and the reliability of text processing can be improved.

[0015] The computer device, computer program product, and computer-readable storage medium provided in the above embodiments belong to the same concept as the corresponding embodiments of the text processing method based on multi-dimensional evaluation, and thus have the same technical effects as the corresponding embodiments of the text processing method based on multi-dimensional evaluation, which will not be elaborated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a structural block diagram of a text generation system in an embodiment.

[0017] Figure 2 It is a structural block diagram of a text generation server in an embodiment.

[0018] Figure 3 It is a flowchart of a text processing method based on multi-dimensional evaluation in an embodiment.

[0019] Figure 4 It is a flowchart of the steps for determining the optimal text from N generated texts in an embodiment.

[0020] Figure 5 It is a flowchart of the steps for determining N target models from M candidate large language models in an embodiment. Detailed implementation manners

[0021] The technical solution of the present invention will be further elaborated in detail below in conjunction with the accompanying drawings of the specification and specific embodiments.

[0022] In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limitations of the present application. All other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present application.

[0023] In the following description, the expression "some embodiments" is involved, which describes a subset of all possible embodiments. It should be noted that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0024] In the following description, the terms "first, second, third" only distinguish similar objects and do not represent a specific order for the objects. It can be understood that "first, second, third" can be interchanged with a specific order or sequence when permitted, so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0025] The present application provides a text generation system, such as Figure 1As shown in the figure, it includes a text generation server 100, multiple large language models 200 accessed by the text generation server 100, and a terminal 300. The terminal 300 can send a text generation request to the text generation server 100 through an application (APP) or a web interface based on the HTTP / HTTPS protocol. In response to the text generation task, the text generation server 100 determines evaluation metrics and weights for evaluating the text generation quality of the text generation task; and calls N target models to output N generated texts. For each generated text of each target model, the text generation server 100 generates evaluation prompt words corresponding to the N generated texts respectively according to the generated text, and each evaluation metric and weight; inputs the evaluation prompt words of the generated text of each target model into the other N - 1 target models among the N target models respectively to obtain cross-evaluation results of each generated text in the other N - 1 target models; determines the optimal text from the N generated texts according to the cross-evaluation results of each generated text as the generation result of the text generation task, and feeds back the generation result to the terminal 300.

[0026] As Figure 2 shown, the system architecture of the text generation server includes:

[0027] The user interface layer is used to receive the text generation task of the terminal and preprocess the text generation task. The user interface layer may include an input processing unit and a preprocessing unit. Among them, the input processing unit is responsible for receiving the text generation task of the terminal. The text generation task supports multi-modal input forms such as text and voice, and supports JSON / XML structured data stream input and unstructured natural language input. The preprocessing unit implements data cleaning and format standardization (such as using an end-to-end ASR model for noise reduction and frame splitting in speech-to-text conversion), and outputs a UTF-8 format text stream with unified encoding. An input cache queue is deployed in this layer, and a double-buffer mechanism is adopted to ensure the throughput in high-concurrency scenarios.

[0028] The intelligent scheduling layer is used to parse the text generation task and schedule the text generation task to determine N target models that can respond to the text generation task. Among them, the intelligent scheduling layer includes an intention understanding module and a dynamic scheduling center. The intention understanding module is used to parse the text generation task and determine the task characteristics of the text generation task. The dynamic scheduling center is a real-time decision-making engine and resource allocator, which is used to determine N target models that match the task characteristics from M candidate large language models

[0029] The model operation layer is used to deploy and access M candidate large language models. In one embodiment, the model operation layer can deploy differentiated large models such as GPT-4, Claude-3.7, Deepseek-R1, Qwen-Max, etc. Each model has an independent computing container and shares the underlying acceleration resources; the parallel computing unit supports the synchronous generation of multiple models and adopts a task segmentation strategy based on tensor parallelism.

[0030] The quality monitoring layer is used to evaluate the generated texts of N target models and determine the optimal text from the N generated texts as the generation result of the text generation task. In one embodiment, the quality monitoring layer can include a cross-evaluation module, an update module, and a feedback closed-loop module. Among them, the cross-evaluation module is used to implement the mutual evaluation mechanism between models; the update module executes a priority dynamic adjustment algorithm to dynamically adjust the model call priority according to the real-time evaluation results; the feedback closed-loop module stores the evaluation results using a MySQL cluster and establishes a model performance map for model optimization.

[0031] As Figure 3 shown, a text processing method based on multi-dimensional evaluation is provided, including:

[0032] Step 302, in response to a text generation task, determine the evaluation metrics and weights for evaluating the text generation quality of the text generation task.

[0033] Some existing methods for evaluating text generation quality usually adopt general metrics (such as BLEU, ROUGE, etc.), which are difficult to accurately evaluate according to the unique requirements of specific tasks, thus reducing the reliability of the generated text.

[0034] In this embodiment, for a text generation task, the evaluation metrics and weights for evaluating the text generation quality of the text generation task are determined, so that the evaluation metrics for the text generation quality of different text generation tasks can be different, and the corresponding evaluation metrics are used for evaluation for different text generation tasks, which is beneficial to improving the reliability of the text.

[0035] In one embodiment, the text generation tasks are classified, and the evaluation metrics for the text generation quality of different types of text generation tasks are different. For example, classifying the text generation tasks can include task categories such as literary creation, contract generation, paper generation, report generation, business copywriting generation, travel guide generation, etc. For text generation tasks such as literary creation, the evaluation can focus on semantic coherence and innovation, and relatively high weights can be set for these two metrics. For the two types of text generation tasks of paper generation and report generation, the evaluation can focus on domain professionalism and logical integrity, and relatively high weights can be set for these two metrics.

[0036] Step 304: Input the text generation requirements of the text generation task into N target models respectively to obtain the generated texts output by the N target models; the target models are large language models.

[0037] Specifically, the text generation server calls the N target models respectively, inputs the text generation requirements of the text generation task into the N target models respectively, and obtains the generated texts output by the N target models.

[0038] For example, for the text generation requirement of "generate a science fiction short story with the theme of quantum time travel, requiring the inclusion of time-space paradox plots, and the language style refers to the works of writer Liu XX", input this text generation requirement into the N target models respectively to obtain N science fiction short stories generated by the N target models according to this text generation requirement.

[0039] Step 306: For the generated text of each target model, generate evaluation prompt words corresponding to the N generated texts respectively according to the generated text, and each evaluation index and weight.

[0040] The evaluation prompt word is the model prompt word used to indicate the model to evaluate the quality of the generated text. This evaluation prompt word is generated according to the generated text, and each evaluation index and weight, and can guide the model to evaluate the quality of the generated text from the specified evaluation index and its weight for the generated text.

[0041] Among them, for the generated text of each target model, corresponding evaluation prompt words are generated to guide the model to evaluate the quality of the generated text of each target model respectively. For example, for the generated texts output by the N target models, corresponding evaluation prompt words are generated respectively.

[0042] In an example, the evaluation prompt word can be: Please score the text "****" in combination with the following evaluation index and its weight. The evaluation index and its weight are: semantic coherence (0.15), domain professionalism (0.15), innovation (0.35), logical integrity (0.25), and format standardization (0.1).

[0043] Step 308: Input the evaluation prompt words of each generated text into the other N - 1 target models among the N target models respectively to obtain the cross - evaluation results of the other N - 1 target models on the generated text.

[0044] In this embodiment, for the N target model response text generation tasks, the generated texts output are evaluated by using a cross-evaluation method among the N target models. Cross-evaluation means that for the generated text of a certain target model, the other N - 1 target models among the N target models are used for evaluation. Through cross-evaluation, the advantages of different target models can be fully utilized, the blind spots of a single model can be compensated, and the reliability of quality evaluation can be improved.

[0045] Suppose there are three target models A, B, and C, then the cross-evaluation method is as follows:

[0046] Generated text Cross-evaluation model adopted Generated text of Model A B and C Generated text of Model B A and C Generated text of Model C A and B

[0047] In this embodiment, by performing cross-evaluation on the generated texts among the target models, for the generated text of each target model, the evaluation results of the other N - 1 target models on the generated text can be obtained. Through cross-evaluation, the advantages of different target models can be fully utilized, the blind spots of a single model can be compensated, and the reliability of quality evaluation can be improved.

[0048] Step 310, according to the cross-evaluation results, determine the optimal text from the N generated texts as the generated result of the text generation task.

[0049] Specifically, according to the cross-evaluation results, the text with the highest quality score in the cross-evaluation can be determined from the N generated texts as the optimal text.

[0050] The above text processing method based on multi-dimensional evaluation uses N target models to execute the text generation task, and for the generated text output by each target model, quality evaluation is performed on the other N - 1 target models to achieve cross-evaluation of the generated text by the N target models. According to the cross-evaluation results, the optimal text is determined from the N generated texts as the generated result of the text generation task. This method determines the evaluation indicators and weights for evaluating the text generation quality of the text generation task, so as to use the corresponding evaluation indicators for evaluation for different text generation tasks, which is beneficial to improving the reliability of the text. Moreover, for the generated texts of the N target models, the optimal text is determined through cross-evaluation. Through cross-evaluation, the advantages of different target models can be fully utilized, the blind spots of a single model can be compensated, and the reliability of quality evaluation can be further improved. By using this method, the text generation quality and the reliability of text processing can be improved.

[0051] In one embodiment, the evaluation prompt is used to indicate that according to the evaluation indicators and weights, the generated text is scored under each evaluation indicator; the cross-evaluation result includes a quality score representing the quality of the generated text, and the quality score is the weighted sum of the scores of the generated text under each evaluation indicator; the optimal text is the text with the highest quality score among the N generated texts.

[0052] In this embodiment, the cross-evaluation result is a quality score. The higher the score, the higher the quality of the generated text. Specifically, the evaluation prompt is used for the index target model to score the generated text under each evaluation index according to the evaluation index and weight.

[0053] In one example, the evaluation indexes and their weights are: semantic coherence (0.15), domain professionalism (0.15), innovation (0.35), logical integrity (0.25), and format standardization (0.1). Then the other N - 1 target models respectively evaluate the generated text from five indexes: semantic coherence, domain professionalism, innovation, logical integrity, and format standardization. Each evaluating target model can output the scores of the generated text under these five indexes, and the sum of the scores under the five indexes and their weights is calculated to obtain the quality score for evaluating the quality of the generated text.

[0054] In this method, the quality evaluation result of the target model for the generated text can be quantified through the quality score, which is convenient for subsequently determining the optimal text from N generated texts. In this embodiment, the optimal text is the text with the highest quality score among the N generated texts. Among them, the highest quality score can include two situations. One situation refers to the highest total quality score evaluated by the N - 1 target models, and the other situation refers to the highest average quality score evaluated by the N - 1 target models.

[0055] In this embodiment, by generating evaluation indicators according to the evaluation index and weight to indicate the model to score the generated text under each evaluation index and calculating the weighted sum to obtain the quality score representing the quality of the generated text, the cross-evaluation result can be reflected by the quality score, that is, the quality of the generated text is quantified, which is convenient for subsequently determining the text with the highest quality score as the optimal text according to the cross-evaluation result.

[0056] In one embodiment, before cross-evaluation, the generated text can also be desensitized. Through desensitization processing, the original generation model information can be hidden to avoid evaluation bias.

[0057] In one embodiment, as Figure 4 shown, determining the optimal text from N generated texts according to the cross-evaluation results of each generated text as the generation result of the text generation task includes:

[0058] Step 402, for the generated text of each target model, establish an n×n-dimensional score matrix for each generated text according to the quality scores of the generated text by the other N - 1 models.

[0059] Specifically, for each generated text, a corresponding scoring matrix is established, which is used to record the quality scores of the generated text by the other N - 1 target models.

[0060] Step 404, after normalizing the scoring matrices of the generated texts, according to the average values of the scoring matrices, the consensus quality scores of the generated texts of the N target models are obtained respectively.

[0061] Specifically, through normalization, the outputs of each model can be normalized to the same scale. Therefore, normalization can eliminate the differences in the scoring scales of individual models. Before normalization, abnormal quality scores can also be filtered through a pre - tuned confidence threshold. In one embodiment, the Z - score standardization method can be used for normalization.

[0062] After normalizing the scoring matrix, the quality scores in the scoring matrix are averaged to obtain the consensus quality scores of the generated texts of the N target models respectively. By taking the matrix of the quality scores after normalization to obtain the consensus quality scores of the generated texts, the deviation of a single model can be smoothed, and the stability of the overall evaluation can be improved.

[0063] Step 406, the generated text with the highest consensus quality score among the N generated sub - texts is determined as the optimal text, and the optimal text is used as the generation result of the text generation task.

[0064] In this embodiment, by constructing the scoring matrix of each generated text, after normalizing the scoring matrix, calculating the average value to obtain the consensus quality score of the generated text, and determining the generated text with the highest consensus quality score among the N generated sub - texts as the optimal text. This method can smooth the deviation of a single model and improve the stability of the overall evaluation by taking the matrix of the quality scores after normalization to obtain the consensus quality score of the generated text.

[0065] In one embodiment, in response to a text generation task, evaluation metrics and weights for evaluating the text generation quality of the text generation task are determined, including: in response to a text generation task, obtaining the text generation requirements of the text generation task; analyzing the text generation requirements to determine the task characteristics of the text generation task; and determining the evaluation metrics and weights for evaluating the text generation quality of the text generation task according to the task characteristics.

[0066] Among them, the text generation task includes text generation requirements. The text generation requirements are the model prompt words, which are a guiding text used to guide a large - language model to generate specific content or perform a task. For example, a text generation requirement is "Generate a science - fiction short story with the theme of quantum time travel, requiring the inclusion of time - space paradox plots, and the language style refers to the works of writer Mr. Liu".

[0067] In one embodiment, the text generation server responds to a text generation task, parses the text requirements of the text generation task, and determines the task characteristics of the text generation task. Among them, the task characteristics can represent at least one or more of the text type, content type, theme type, and style label to be generated.

[0068] In one embodiment, by parsing the text generation requirements, multi-level task categories can be determined, and task characteristics can be obtained based on the multi-level task categories. Among them, the multi-level task category is a structured and extensible classification framework that decomposes complex tasks into subtasks with clear boundaries and relevance through a hierarchical relationship. Among them, the hierarchical depth of the multi-level task category can be defined according to the actual task and is not limited here. For text generation tasks, the multi-level task category can include first-level (major category), second-level (subcategory), third-level (style) - fourth-level (plot setting). For example, the multi-level task category includes at least two of the text type, content type, theme type, and style label. Taking the text generation requirement "Write the introduction part of the research on graphene-perovskite composite photovoltaic materials, which needs to include a literature review in the past three years and meet the requirements of the ACS Nano format" as an example, its multi-level task category can be determined to include: literary creation, short story, science fiction, the style of writer Liu XX, and time-space paradox plot.

[0069] According to the task characteristics, evaluation indicators and weights for evaluating the text generation quality of the text generation task are determined. Thus, the evaluation indicators for the text generation quality of text generation tasks with different task characteristics are different. Evaluating text generation tasks with different task characteristics using their corresponding evaluation indicators is beneficial to improving the reliability of the text.

[0070] In one embodiment, the evaluation indicators include at least two of semantic coherence, domain professionalism, innovation, logical integrity, and format standardization.

[0071] In one embodiment, the text generation tasks are classified. For different types of text generation tasks, the evaluation indicators for their text generation quality are different. For example, classifying text generation tasks can include task categories such as literary creation, contract generation, paper generation, report generation, business copywriting generation, and travel guide generation. For text generation tasks such as literary creation, evaluation can focus on semantic coherence and innovation, and higher weights are set for these two indicators. For the two types of text generation tasks of paper generation and report generation, evaluation can focus on domain professionalism and logical integrity, and higher weights are set for these two indicators.

[0072] In the specific implementation method, a quality evaluation indicator mapping table corresponding to different types of text generation tasks can be established, and the quality evaluation indicators and their weights corresponding to the text generation task type can be determined by looking up the table.

[0073] In this embodiment, evaluation metrics and weights for evaluating the quality of text generation for this text generation task are determined according to the task characteristics, so as to adopt corresponding evaluation metrics for different text generation tasks for evaluation, which is beneficial to improving the reliability of the text.

[0074] In one embodiment, the method further includes: determining N target models that match the task characteristics from M candidate large language models; where 1 ≤ N < M.

[0075] Specifically, the candidate large language models are the models accessed by this system. This system can deploy M currently popular large language models, and determine N target models that match the task characteristics from the candidate large language models. Among them, the number of N can be set according to actual business needs, such as setting N = 3.

[0076] In one embodiment, this system deploys differentiated large models such as GPT-4, Claude-3.7, Deepseek-R1, and Qwen-Max. Each model has an independent computing container and shares underlying acceleration resources; this system supports multi-model synchronous generation and adopts a task segmentation strategy based on tensor parallelism.

[0077] In one embodiment, the ability characteristics of each large language model are analyzed in advance, and ability characteristics are set for each large language model according to the ability of each large language. In one embodiment, according to the matching degree between the ability characteristics of the candidate large language models and the task characteristics, the top N target models are determined, so that large language models with domain expertise can be preferentially matched for this text generation task.

[0078] In one embodiment, N target models are determined from the candidate large language models according to historical performance, the similarity between the task characteristics and the model ability characteristics, and resource energy consumption. In this embodiment, in addition to considering the matching degree between the ability characteristics of the large language model and the task characteristics, the historical performance and resource consumption of each large language model are also considered.

[0079] In one embodiment, weights for historical performance, the matching degree between the task and the model, and resource energy consumption are preset respectively. Scores are given to each candidate large language model respectively from historical performance, the similarity between the task characteristics and the model ability characteristics, and resource energy consumption. According to the weights and the scores of each dimension, the total scores of each candidate large language model are calculated, and the top N models are selected as the target models according to the total score ranking.

[0080] In this embodiment, dynamically matching N target models from M candidate large language models to execute this text generation task can give full play to the differentiated advantages of different large language models in different tasks and improve the quality of text generation.

[0081] In one embodiment, determining N target models that match the task characteristics from M candidate large language models includes: obtaining the capability characteristics of the M candidate large language models; and determining N target models from the M candidate large language models based on at least one of the historical performance and resource energy consumption of the M candidate large language models, as well as the matching degree between the capability characteristics of each large language model and the task characteristics.

[0082] Among them, the historical performance refers to whether the large language model has executed this type of text generation task in a recent period of time, and performances such as being adopted as the optimal text. In actual operation, a time decay window can be established, with recent task performances having higher weights, and at the same time, setting the longest traceable period to prevent data from becoming outdated.

[0083] Among them, the resource energy consumption refers to the computing time, cost, and hardware resource consumption of the large language model. When the quality is comparable, a lightweight model is preferred.

[0084] Among them, the matching degree with the task refers to the matching degree between the capabilities of the large language model and the task. Analyze the capability characteristics of each large language model in advance, and set capability characteristics for each large language model according to the capabilities of each large language. In one embodiment, determine the matching degree with the task according to the matching degree between the capability characteristics of the candidate large language model and the task characteristics.

[0085] In this embodiment, it is possible to dynamically match N target models from multiple large language models to execute this text generation task based on dimensions such as historical performance, resource energy consumption, and matching degree with the task, so as to achieve the optimal allocation of model resources.

[0086] Specifically, determine N target models from the M candidate large language models based on at least one of the historical performance and resource energy consumption of the M candidate large language models, as well as the matching degree between the capability characteristics of each large language model and the task characteristics, as Figure 5 shown, including:

[0087] Step 502, determine the historical performance scores of each candidate large language model according to the historical performance of the M candidate large language models.

[0088] Among them, the historical performance refers to whether the large language model has executed this type of text generation task in a recent period of time, and performances such as being adopted as the optimal text. In actual operation, the scoring method for historical performance can establish a time decay window, with recent task performances having higher weights. For example, the score of a large language model with the above historical performance within one day is 10 points, and the score of a large language model with the above historical performance within one week is 5 points. Determine the historical performance scores of each candidate large language model according to the historical performance of the M candidate large language models and the scoring method.

[0089] Step 504: Determine the matching scores of each candidate large language model according to the matching degree between the ability characteristics and task characteristics of each candidate large language model.

[0090] Specifically, pre-analyze the ability characteristics of each large language model, and set the ability characteristics for each large language model according to the capabilities of each large language model. In one embodiment, according to the matching degree between the ability characteristics and task characteristics of the candidate large language model, determine the matching degree with the task. Among them, the matching degree is directly proportional to the matching score, and the higher the matching degree, the higher the matching score. For example, if the matching degree is 100%, the matching score can be 100 points.

[0091] Step 506: Determine the energy consumption scores of each candidate large language model according to the resource energy consumption of each candidate large language model.

[0092] Resource energy consumption refers to the computing time-consuming, cost, and hardware resource consumption of the large language model. Among them, the resource energy consumption of each candidate large language model can be determined in advance according to the computing time-consuming, cost, and hardware resource consumption of the large language model. It can also be updated regularly according to the computing time-consuming, cost, and hardware resources within a period of time to ensure the timeliness of the resource energy consumption evaluation.

[0093] Step 508: Perform a weighted summation process on the historical performance scores, matching scores, and energy consumption scores of each candidate large language model according to the evaluation weights to obtain the scores of each candidate large language model.

[0094] Specifically, perform a weighted summation process on the historical performance scores, matching scores, and energy consumption scores of each candidate large language model according to the weights of historical performance, resource energy consumption, and matching degree with the task to obtain the scores of each candidate large language model.

[0095] In one embodiment, the formula for calculating the score of the candidate large language model is:

[0096] PriorityScore(t)_m = α * S(t)_hist + β * C(t)_eval + γ * D(t)_domain

[0097] Where: m is the model identifier, m ∈ {1, 2,..., M}, t is the time period serial number, α is the evaluation weight of historical performance, β is the evaluation weight of the matching degree with the task, γ is the evaluation weight of resource energy consumption, α + β + γ = 1; S(t)_hist is the historical performance score, C(t)_eval is the matching score, D(t)_domain is the energy consumption score;

[0098] Among them, S(t) hist= (∑(k = 1 to k)^λ(t - k) * R(k)_m / ∑(k = 1 to k)λ^(t - k)), where λ is the time decay factor, λ ∈ (0, 1), and R(k)_m is the score of the model in the k-th period.

[0099] Step 510, sort the M candidate large language models according to their scores, and select the top N target models with the highest rankings.

[0100] Specifically, sort each candidate large language model according to its score, and select the top N target models with the highest scores from the M candidate large language models. N can be flexibly set according to requirements. In one embodiment, N is 3, then select the top three large language models with the highest scores from the M candidate large language models as the target models.

[0101] In this embodiment, the candidate large language models are sorted according to dimensions such as historical performance, resource consumption, and task matching degree to achieve the optimal allocation of model resources.

[0102] The text processing method based on multi-dimensional evaluation of this application has the following technical effects:

[0103] 1. For text generation tasks, determine the evaluation indicators and weights for evaluating the text generation quality of the text generation task, so as to use the corresponding evaluation indicators for evaluation for different text generation tasks, which is beneficial to improving the reliability of the text.

[0104] 2. Determine the optimal text for the generated texts of the N target models through cross-evaluation. Through cross-evaluation, the advantages of different target models can be fully utilized to make up for the blind spots of a single model, and further improve the reliability of quality evaluation. By using this method, the text generation quality and the reliability of text processing can be improved.

[0105] 3. Develop a model priority algorithm that integrates the time decay factor, domain adaptability, and resource consumption to achieve the balanced optimization of historical performance, real-time ability, and resource consumption, and achieve the optimal allocation of model resources.

[0106] On the other hand, this embodiment of the application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it realizes each process of the above-mentioned embodiment of the text processing method based on multi-dimensional evaluation and can achieve the same technical effects. To avoid repetition, it will not be elaborated here. Among them, the computer-readable storage medium, such as Read-Only Memory (ROM for short), Random Access Memory (RAM for short), magnetic disk or optical disk, etc.

[0107] In another aspect of the embodiments of the present application, a computer device is further provided, which includes a processor and a memory connected to the processor. A computer program executable by the processor is stored on the memory. When the computer program is executed by the processor, it implements each process of the text processing method embodiment based on multi-dimensional evaluation and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0108] In another aspect of the embodiments of the present application, a computer program product is further provided, including a computer program. When the computer program is executed by a processor, it implements each process of the text processing method embodiment based on multi-dimensional evaluation and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0109] It should be noted that in this article, the term "including", "comprising" or any other variant thereof is intended to cover a non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including that element.

[0110] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc) and includes several instructions for causing a terminal (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0111] The above are only the specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application, and all should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A text processing method based on multi-dimensional evaluation, characterized in that The method includes: In response to a text generation task, determining evaluation metrics and weights for evaluating the text generation quality of the text generation task; Inputting the text generation requirements of the text generation task into N target models respectively to obtain generated texts output by the N target models; the target models are large language models; For the generated text of each target model, according to the generated text, as well as each evaluation metric and weight, generating evaluation prompt words corresponding to the N generated texts respectively; Inputting the evaluation prompt words of each generated text into the other N-1 target models among the N target models respectively to obtain cross-evaluation results of the other N-1 target models on the generated text; According to the cross-evaluation results, determining the optimal text from the N generated texts as the generation result of the text generation task.

2. The method according to claim 1, wherein The evaluation prompt words are used to indicate scoring the generated text under each evaluation metric according to the evaluation metrics and weights; the cross-evaluation results include quality scores representing the quality of the generated text, and the quality score is the weighted sum of the scores of the generated text under each evaluation metric; the optimal text is the text with the highest quality score among the N generated texts.

3. The method according to claim 2, wherein The determining the optimal text from the N generated texts according to the cross-evaluation results of each generated text as the generation result of the text generation task includes: For the generated text of each target model, according to the quality scores of the generated text by the other N-1 models, establishing an n×n dimensional scoring matrix for each generated text; After normalizing the scoring matrices of each generated text, according to the average values of each scoring matrix, obtaining the consensus quality scores of the generated texts of the N target models respectively; Determining the generated text with the highest consensus quality score among the N generated texts as the optimal text, and using the optimal text as the generation result of the text generation task.

4. The method according to claim 1, characterized in that The responding to a text generation task and determining evaluation metrics and weights for evaluating the text generation quality of the text generation task includes: In response to a text generation task, obtaining the text generation requirements of the text generation task; Analyzing the text generation requirements to determine the task characteristics of the text generation task; According to the task characteristics, determining evaluation metrics and weights for evaluating the text generation quality of the text generation task.

5. The method according to any one of claims 1 to 4, characterized in that, The evaluation metrics include at least two of semantic coherence, domain expertise, innovation, logical integrity, and format standardization.

6. The method according to claim 4, wherein The method further includes: Determining N target models that match the task characteristics from M candidate large language models; where 1≤N<M.

7. The method according to claim 6, wherein The determining N target models that match the task characteristics from M candidate large language models includes: Obtaining the ability characteristics of the M candidate large language models; Determining N target models from the M candidate large language models according to at least one of the historical performance and resource consumption of the M candidate large language models, and the matching degree between the ability characteristics of each large language model and the task characteristics.

8. A computer device, characterized in that, It includes a processor and a memory connected to the processor. A computer program executable by the processor is stored on the memory. When the computer program is executed by the processor, it implements the steps of the text processing method based on multi-dimensional evaluation as described in any one of claims 1 to 7.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the text processing method based on multi-dimensional evaluation as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium. When the computer program is executed by the processor, it implements the steps of the text processing method based on multi-dimensional evaluation as described in any one of claims 1 to 7.

Images