Text evaluation method and device, equipment and medium
Through the text evaluation method, the target text is evaluated using the preset large language model, which solves the problems of high cost of manual evaluation and insufficient objectivity, realizes automated and efficient text evaluation, and improves the objectivity and professionalism of the evaluation results.
Patent Information
- Application Number
- CN202411014557.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-26
- Publication Date
- 2024-12-03
AI Technical Summary
In the prior art, evaluating the answer text output from the large language model requires manual completion, resulting in high labor and time costs, and insufficient objectivity, diversity and professionalism of the evaluation process.
Provide a text evaluation method, by obtaining the question text and answer text corresponding to the target text, inputting it into the preset large language model for analysis using preset evaluation criteria, obtaining multiple candidate evaluation results, and filtering and fusion according to preset conditions , generate target evaluation results.
Automatic evaluation of the answer text output from large language models is realized, which reduces the time and labor cost of manual evaluation and ensures the objectivity, diversity and professionalism of the evaluation results.
Smart Images

Figure CN119066378A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a text evaluation method, device, electronic device and storage medium. Background Art
[0002] The emergence of large language models (LLMs) makes it possible to analyze complex and long texts. They can be applied to the deconstruction, analysis, and evaluation of complex and long texts such as plots, scripts, and various literary works.
[0003] For example, a large language model can ask questions to a long complex text, get answers to the questions by analyzing the long complex text, and output the corresponding answer text. Then, the answer text output by the large language model can be evaluated. If the answer text is of high quality, it can indicate to a certain extent that the content, structure and other information of the long complex text are comprehensive and clear. If the quality of the answer text is poor, it can indicate to a certain extent that the long complex text has missing content, information or structural problems. In this way, by evaluating the answer text, the evaluation of the long complex text can be achieved.
[0004] However, in the prior art, the evaluation of answer texts often needs to be done manually, and the evaluators are required to have background knowledge and experience in different fields. Due to the large amount of complex and long texts, the manpower and time costs required for manual evaluation are also relatively high. Moreover, the same person cannot evaluate from multiple dimensions or without subjectivity, which will lead to objectivity risk costs of the evaluation results. Different people have different analysis standards, which will introduce uniformity risk costs of evaluation standards and analysis dimensions, etc. Therefore, there is an urgent need for an efficient method for evaluating answer texts output by large language models with unified evaluation standards to help realize the evaluation of complex and long texts. Summary of the invention
[0005] The purpose of the embodiments of the present application is to provide a text evaluation method, apparatus, device and storage medium, which can solve the problems in the prior art that the evaluation of the answer text output by a large language model often needs to be completed manually, with high labor and time costs, and the objectivity, diversity and professionalism of the evaluation process are not high enough.
[0006] In a first aspect, an embodiment of the present application provides a text evaluation method, comprising:
[0007] Obtaining a text to be evaluated; the text to be evaluated includes a question text, an answer text, and a preset evaluation standard corresponding to a target text, wherein the answer text is obtained by analyzing the target text based on the question text by a text analysis model, and the preset evaluation standard is an evaluation standard used to evaluate the question text and the answer text corresponding to the target text;
[0008] Inputting the text to be evaluated into a preset large language model, analyzing the text to be evaluated, and obtaining a plurality of candidate evaluation results; the candidate evaluation results include evaluation contents and evaluation scores of the question text and its corresponding answer text based on the preset evaluation criteria;
[0009] The candidate evaluation results whose evaluation scores meet the preset conditions are used as reference evaluation results;
[0010] The reference evaluation results are fused to obtain the target evaluation results.
[0011] Optionally, obtaining the text to be evaluated includes:
[0012] Obtain the question text, answer text and preset evaluation criteria corresponding to the target text;
[0013] The question text, the answer text and the preset evaluation criteria are filled into a preset template to obtain a text to be evaluated.
[0014] Optionally, the step of obtaining a question text, an answer text, and a preset evaluation standard corresponding to the target text includes:
[0015] Obtaining a target text and a preset evaluation standard corresponding to the target text;
[0016] Based on the target text, obtaining a corresponding question text;
[0017] The question text is input into a text analysis model, and the text analysis model analyzes the target text based on the question text to obtain an answer text.
[0018] Optionally, before inputting the text to be evaluated into a preset large language model and analyzing the text to be evaluated to obtain a plurality of candidate evaluation results, the method further includes:
[0019] Obtaining a sample evaluation text and a sample evaluation result, wherein the sample evaluation text includes a sample question, a sample answer, and a sample evaluation standard corresponding to the sample text;
[0020] Filling the sample data and the sample evaluation result into the preset template to obtain a sample template text;
[0021] The text to be evaluated is input into a preset large language model, and the text to be evaluated is analyzed to obtain multiple candidate evaluation results, including:
[0022] The text to be evaluated and the example template text are input into a preset large language model, context learning is performed on the text to be evaluated and the example template text, and based on the mapping relationship between the example evaluation text and the example evaluation result, a plurality of candidate evaluation results corresponding to the text to be evaluated are determined.
[0023] Optionally, the text to be evaluated is input into a preset large language model, and the text to be evaluated is analyzed to obtain multiple candidate evaluation results, including:
[0024] The text to be evaluated is input into a preset large language model, and the preset large language model analyzes the text to be evaluated to obtain multiple initial evaluation results and corresponding prediction probabilities, and based on the order of the prediction probabilities from large to small, a preset number of candidate evaluation results are determined and output from the initial evaluation results.
[0025] Optionally, taking the candidate evaluation result whose evaluation score meets a preset condition as a reference evaluation result includes:
[0026] Determining a score threshold based on the candidate evaluation results and the corresponding evaluation scores;
[0027] The candidate evaluation results whose evaluation scores are greater than the score threshold are used as reference evaluation results.
[0028] Optionally, fusing the reference evaluation results to obtain a target evaluation result includes:
[0029] The reference evaluation result is input into the preset large language model, and the preset large language model performs text fusion on the reference evaluation result to obtain a target evaluation result.
[0030] Optionally, the preset evaluation criteria include multiple evaluation dimensions, and each of the candidate evaluation results corresponds to the multiple evaluation dimensions.
[0031] In a second aspect, an embodiment of the present application provides a text evaluation device, comprising:
[0032] An acquisition module is used to acquire a text to be evaluated; the text to be evaluated includes a question text, an answer text and a preset evaluation standard corresponding to the target text, wherein the answer text is obtained by analyzing the target text based on the question text by a text analysis model, and the preset evaluation standard is an evaluation standard used to evaluate the question text and the answer text corresponding to the target text;
[0033] An analysis module, used for inputting the text to be evaluated into a preset large language model, analyzing the text to be evaluated, and obtaining a plurality of candidate evaluation results; the candidate evaluation results include evaluation contents and corresponding evaluation scores of the question text and its corresponding answer text based on the preset evaluation criteria;
[0034] A screening module is used to select the candidate evaluation results whose evaluation scores meet the preset conditions as reference evaluation results;
[0035] The fusion module is used to fuse the reference evaluation results to obtain the target evaluation results.
[0036] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.
[0037] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.
[0038] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.
[0039] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.
[0040] In an embodiment of the present application, a text to be evaluated is obtained; the text to be evaluated includes a question text corresponding to a target text, an answer text obtained by analyzing the target text based on the question text by a text analysis model, and a preset evaluation standard adopted for evaluating the question text and the answer text corresponding to the target text; the text to be evaluated is input into a preset large language model, and the text to be evaluated is analyzed to obtain a plurality of candidate evaluation results; the candidate evaluation results include evaluation contents and evaluation scores of the question text and its corresponding answer text based on the preset evaluation standards; the candidate evaluation results whose evaluation scores meet the preset conditions are used as reference evaluation results; the reference evaluation results are fused to obtain a target evaluation result.
[0041] In this way, the preset large language model can evaluate the answer text in the text to be evaluated based on the preset evaluation criteria and question text in the text to be evaluated, wherein the answer text is obtained by the text analysis model answering the question text based on the target text. It can be understood that if the target text is of high quality, then the text analysis model can extract the information required to answer the question text from the target text more comprehensively, thereby obtaining a high-quality answer text. On the contrary, if the target text is of poor quality, then the text analysis model cannot extract the required information from the target text, and the quality of the answer text obtained will be relatively poor. Therefore, the target evaluation result is not only an evaluation of the answer text, but also reflects the quality of the target text. Moreover, in the evaluation process of the text to be evaluated, it is automatically completed by the preset large language model, which does not rely on manual labor, and can reduce the time and labor cost of manual evaluation of answers. At the same time, the preset large language model is evaluated according to the preset evaluation criteria, and the analysis criteria and analysis dimensions in the evaluation process are consistent, and the target evaluation results obtained have high objectivity, diversity and professionalism. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 is a flowchart of a text evaluation method according to an exemplary embodiment;
[0043] Figure 2 is a logic diagram of a text evaluation method according to an exemplary embodiment;
[0044] Figure 3 is a schematic diagram of a text analysis framework according to an exemplary embodiment;
[0045] Figure 4 is a block diagram of a text evaluation device according to an exemplary embodiment;
[0046] Figure 5 is a block diagram of a base station text evaluation electronic device according to an exemplary embodiment;
[0047] Figure 6 It is a block diagram of a device for text evaluation according to an exemplary embodiment. DETAILED DESCRIPTION
[0048] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.
[0049] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0050] The text evaluation method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.
[0051] Large language models can be applied to the deconstruction, analysis, and evaluation of complex long texts such as plots, scripts, and various literary works. For example, through the large language model, you can ask questions to complex long texts, get answers to the questions, and output the corresponding answer texts. Then, you can evaluate the answer texts output by the large language model. If the answer text is of high quality, then it can be explained to a certain extent that the content, structure, and other information of the complex long text are comprehensive and clear. If the quality of the answer text is poor, it can be shown to a certain extent that the complex long text has missing content, information, or structural problems. In this way, by evaluating the answer text, you can achieve the evaluation of complex long texts.
[0052] However, in related technologies, the evaluation of answer texts often needs to be done manually, which requires evaluators to have background knowledge and experience in different fields. Due to the large amount of complex and long texts, the manpower and time costs required for manual evaluation are also relatively high. Moreover, the same person cannot evaluate from multiple dimensions or without subjectivity, which will lead to objectivity risk costs of the evaluation results. Different people have different analysis standards, which will introduce risk costs of the uniformity of evaluation standards and analysis dimensions, etc.
[0053] Based on this, the present application provides a text evaluation method to solve the above problems. The method does not need to rely on manual work, is efficient and has a unified evaluation standard, and can help to evaluate complex and long texts.
[0054] Specifically, the present application can be applied to a server, a local server or other servers in the network, to obtain the text to be evaluated and perform subsequent text evaluation steps to obtain multiple candidate evaluation results, and then obtain the target evaluation result by screening and fusing the candidate evaluation results, so as to realize automatic evaluation of the target text based on a large language model and reduce the manpower and time costs required for text evaluation.
[0055] The above-mentioned server can be a server cluster or distributed system composed of multiple physical servers, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), as well as big data and artificial intelligence platforms.
[0056] Among them, cloud technology refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to realize data computing, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, which is used on demand and flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, in the future, each item may have its own identification mark, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately. All kinds of industry data require strong system backing support, which can only be achieved through cloud computing.
[0057] Figure 1 The flowchart of a text evaluation method according to an exemplary embodiment is shown. The text evaluation method includes the following steps.
[0058] In step S11, the text to be evaluated is obtained; the text to be evaluated includes a question text, an answer text and a preset evaluation standard corresponding to the target text, wherein the answer text is obtained by analyzing the target text based on the question text by a text analysis model, and the preset evaluation standard is the evaluation standard used to evaluate the question text and the answer text corresponding to the target text.
[0059] In this step, you first need to obtain the text to be evaluated, which includes the question text, the answer text, and the preset evaluation criteria. The question text refers to the question asked to the target text; the answer text is the answer obtained by the text analysis model based on the question text to the target text; the preset evaluation criteria correspond to the target text.
[0060] The preset evaluation criteria refer to the evaluation criteria used when evaluating the question text and answer text corresponding to the target text, which may include multiple evaluation dimensions, each of which may correspond to different evaluation criteria, and this application does not specifically limit this. The preset evaluation criteria can be set in advance according to the type of the target text. For example, if the target text is a script, then when evaluating the script, it can be evaluated from different evaluation dimensions such as plot and characters. In the plot dimension, the integrity of elements such as cause, process, climax, and ending needs to be used as evaluation criteria. In the character dimension, the integrity of elements such as life experience, life experience, and personality transformation needs to be used as evaluation criteria, and so on. Through the establishment of preset evaluation criteria, the target text can be analyzed and evaluated more specifically and comprehensively, thereby obtaining more accurate target evaluation results.
[0061] In the present application, the target text is a complex and long text that the text analysis model needs to analyze and answer, which can be a script, novel, paper, etc. After determining the question text, the text analysis model can ask questions to the target text based on the question text to obtain the answer text. Among them, the text analysis model can adopt any model that can perform semantic analysis on the target text, such as a large language model, a natural language processing model or a neural network model, and the present application does not limit this.
[0062] In one implementation, obtaining the text to be evaluated includes:
[0063] Obtain the question text, answer text and preset evaluation criteria corresponding to the target text; fill the question text, answer text and preset evaluation criteria into the preset template to obtain the text to be evaluated.
[0064] Among them, it can be seen from the above that the answer text is obtained by analyzing the target text based on the question text by the text analysis model, and the preset evaluation standard is the evaluation standard used to evaluate the question text and the answer text corresponding to the target text.
[0065] In other words, the text to be evaluated can adopt a unified preset template to make it more standardized. In this way, in the subsequent processing process, the text to be evaluated can be directly used as the prompt word input of the preset large language model, which is convenient for the preset large language model to understand and obtain more accurate target evaluation results.
[0066] For example, Figure 2As shown, it is a logical diagram of the text evaluation method in the present application, wherein the preset template is marked with the string "Thoughts:", followed by the template content and the question text to be filled in. For example, the "[…]" after "question" can be replaced with the question text related to the target text, the answer text can be inserted after "the answer", and the "[…]" after "criteria of" can be replaced with the preset evaluation criteria. In this way, the complete text to be evaluated is obtained.
[0067] Alternatively, the answer text can be independent of Figure 2 The character string including the question text and the preset evaluation criteria is shown to generate two sections of text to be evaluated in the form of a dialogue context, which is not limited in this application.
[0068] In the present application, there may be multiple ways to describe the preset template and fill in the dynamically changing content, which can be adjusted to achieve the best return result of the preset large language model. The present application does not specifically limit this.
[0069] In one implementation, obtaining a question text, an answer text, and a preset evaluation standard corresponding to a target text includes:
[0070] Obtain a target text and a preset evaluation standard corresponding to the target text; obtain a corresponding question text based on the target text; input the question text into a text analysis model, and the text analysis model analyzes the target text based on the question text to obtain an answer text.
[0071] That is to say, in the present application, the target text can be obtained first, and then the question text corresponding to the target text can be obtained. Then, the text analysis model analyzes the target text to obtain the answer corresponding to the question text as the answer text, wherein the question text can be automatically generated by the text analysis model or other models, that is, by asking and answering the target text, the question text and the answer text are generated, or the question text can also be manually input, which is not limited in the present application.
[0072] In this way, the question text that asks questions about the target text is itself based on the target text, and can better match the target text, so as to achieve targeted questioning of the target text. As a result, the answer text obtained can more completely and appropriately reflect the content included in the target text, thereby making the final target evaluation result more accurate.
[0073] For example, the target text may be a script, and the question text is the question asked about the script, such as asking about the plot outline of the script or a biography of a character in the script. The answer text is the answer obtained by the text analysis model after analyzing the target text based on the question text. The preset evaluation criteria are the evaluation criteria used to evaluate the question text and the answer text. For example, the evaluation of the script may include two dimensions: characters and plot, and each dimension needs to be evaluated for element completeness separately, and so on.
[0074] Among them, the text analysis model can be any model that can be used for text analysis, or the text analysis model can be the same model as the preset large language model for subsequent evaluation of the text to be evaluated. In other words, only the preset large language model is needed to realize the whole process of asking, answering and evaluating the target text. In this way, the difficulty of model calling can be reduced, the text evaluation process can be simpler, the preset large language model can be fully utilized, and the cost of model training can be reduced.
[0075] In step S12, the text to be evaluated is input into a preset large language model, and the text to be evaluated is analyzed to obtain multiple candidate evaluation results; the candidate evaluation results include evaluation contents and corresponding evaluation scores of the question text and its corresponding answer text based on preset evaluation criteria.
[0076] In this step, the text to be evaluated is input into a preset large language model. The preset large language model can evaluate the question text and the answer text based on preset evaluation criteria to obtain multiple possible candidate evaluation results, thereby achieving a preliminary evaluation of the text to be evaluated.
[0077] For example, if the preset evaluation criteria include multiple evaluation dimensions, then each candidate evaluation result corresponds to multiple evaluation dimensions. Therefore, the analysis criteria and analysis dimensions of the candidate evaluation results in the evaluation process are consistent, which is conducive to achieving the objectivity of the final target evaluation results.
[0078] Among them, the candidate evaluation results include evaluation content and corresponding evaluation scores. The evaluation content refers to the evaluation description of the question text and the answer text. For example, if the target text is a script, the question text is a summary of the script content, and the preset evaluation criteria include plot rationality, character rationality, etc., then the evaluation content can include whether the plot in the script is complete and clear, whether the character development is logical, and so on.
[0079] In one implementation, the text to be evaluated is input into a preset large language model, and the text to be evaluated is analyzed to obtain multiple candidate evaluation results, including:
[0080] The text to be evaluated is input into a preset large language model, which analyzes the text to be evaluated to obtain multiple initial evaluation results and corresponding prediction probabilities, and based on the order of prediction probabilities from large to small, a preset number of candidate evaluation results are determined and output from the initial evaluation results.
[0081] That is to say, after the text to be evaluated is delivered to the preset large language model, the preset large language model can be used to implement beam search decoding, and the top k initial evaluation results with the largest prediction probabilities generated by the preset large language model are returned as candidate evaluation results, where k is a preset number. Beam search is a screening method that can find the optimal solution in a relatively limited search space at a relatively low cost, and the solution obtained is close to the optimal solution in the entire search space.
[0082] Among them, the prediction probability refers to the probability that the text to be evaluated matches the initial evaluation result obtained after the preset large language model analyzes the text to be evaluated. That is to say, the larger the prediction probability, the more accurate the initial evaluation result, and the more it can reflect the true characteristics of the text to be evaluated. Conversely, the smaller the prediction probability, the more inaccurate the initial evaluation result, and it cannot accurately reflect the true characteristics of the text to be evaluated.
[0083] In this way, by screening a preset number of initial evaluation results with the largest prediction probabilities, some initial evaluation results with smaller prediction probabilities can be eliminated, and the retained candidate evaluation results can better reflect the true characteristics of the text to be evaluated, and thus, the final target evaluation results will be more accurate.
[0084] In one implementation, before inputting the text to be evaluated into a preset large language model and analyzing the text to be evaluated to obtain multiple candidate evaluation results, the method further includes:
[0085] Obtaining sample evaluation text and sample evaluation results, wherein the sample evaluation text includes sample questions, sample answers, and sample evaluation criteria corresponding to the sample text;
[0086] Fill the sample data and sample evaluation results into the preset template to obtain the sample template text;
[0087] Correspondingly, the text to be evaluated is input into the preset large language model, and the text to be evaluated is analyzed to obtain multiple candidate evaluation results, including:
[0088] The text to be evaluated and the example template text are input into a preset large language model, context learning is performed on the text to be evaluated and the example template text, and multiple candidate evaluation results corresponding to the text to be evaluated are determined based on the mapping relationship between the example evaluation text and the example evaluation result.
[0089] That is to say, the example evaluation text and the example evaluation results can also be filled into a preset template that is consistent with the text to be evaluated to make it more standardized. Based on the example evaluation text, the gaps corresponding to the question text, answer text and preset evaluation criteria in the above preset template are replaced, and the example evaluation results are filled into the corresponding evaluation score "Score" and evaluation content "Reasoning". This is used to guide the preset large language model to perform contextual learning, so that the preset large language model can better complete the evaluation reasoning of the text to be evaluated in the form of few-shots.
[0090] In step S13, the candidate evaluation results whose evaluation scores meet the preset conditions are used as reference evaluation results.
[0091] In this step, the candidate evaluation results output by the preset large language model can be further screened, and one or more that meet the preset conditions can be selected as reference evaluation results. By further optimizing the multiple candidate evaluation results obtained, a more representative part can be screened out, thereby further improving the accuracy of the final target evaluation result.
[0092] Among them, the candidate evaluation results whose evaluation scores meet the preset conditions may refer to the candidate evaluation results whose evaluation scores are greater than the preset threshold, or, may refer to the candidate evaluation results whose evaluation scores are ranked in the top few, or, may refer to several candidate evaluation results whose evaluation scores are closest to the median, and so on. The preset conditions can be set specifically according to different scenarios or requirements, and this application does not limit this.
[0093] In one implementation, the candidate evaluation results whose evaluation scores meet the preset conditions are used as reference evaluation results, including:
[0094] Based on the candidate evaluation results and the corresponding evaluation scores, a score threshold is determined; and the candidate evaluation results with evaluation scores greater than the score threshold are used as reference evaluation results.
[0095] That is to say, a score threshold can be determined first through an election mechanism based on the evaluation scores of each candidate evaluation result. For example, a threshold selection can be performed on the evaluation scores corresponding to the candidate evaluation results based on the minimum risk criterion, maximum likelihood criterion, etc., as the score threshold; or, the average, median or mode of the evaluation scores corresponding to the candidate evaluation results can be determined as the score threshold, and the present application does not make any specific limitations on this.
[0096] Then, the candidate evaluation results whose evaluation scores are greater than the score threshold are used as reference evaluation results for subsequent generation of the final target evaluation results. In this way, the reference evaluation results can more accurately reflect the average performance of multiple evaluations of the text to be evaluated, compared with the candidate evaluation results, which is conducive to improving the accuracy of the final target evaluation results.
[0097] In step S14, the reference evaluation results are fused to obtain the target evaluation results.
[0098] In this step, the reference evaluation results obtained in the previous step can be integrated. By reintegrating the evaluation text and comprehensively calculating the evaluation scores, a target evaluation result with smoother expression and more reasonable scores can be obtained.
[0099] Among them, the target evaluation result includes two parts: target evaluation content and target evaluation score. The target evaluation score can be the average of each evaluation score corresponding to the reference evaluation result, or the corresponding weight is determined based on the evaluation score of the reference evaluation result, and then the evaluation score of the reference evaluation result is weighted and summed to obtain the target evaluation score, etc. This application does not make any specific limitations on this.
[0100] The target evaluation results can reflect the performance of the answer text under the preset evaluation criteria, and the performance of the answer text can reflect whether the target text can provide all the information needed to answer the question. Furthermore, the quality of the target text can be judged through the target evaluation results.
[0101] In one implementation, the reference evaluation results are integrated to obtain the target evaluation results, including:
[0102] The reference evaluation results are input into the preset large language model, and the preset large language model performs text fusion on the reference evaluation results to obtain the target evaluation results.
[0103] That is to say, new prompt words can be generated based on the retained reference evaluation results as the input of the preset large language model. Then, the preset large language model fuses the reference evaluation results, organically integrating multiple reference evaluation results into one to obtain the target evaluation result.
[0104] In this way, by utilizing the preset large language model, there is no need to introduce a new model for fusion processing in the process of fusion of reference evaluation results, which further improves the efficiency of the text evaluation method and saves the computing resources required to obtain or train new models.
[0105] In addition, other models may also be used to integrate the reference evaluation results to obtain the target evaluation results, and this application does not specifically limit this.
[0106] Below Figure 2 Taking the illustrated embodiment as an example, the text evaluation method proposed in the present application is described. First, the question text, the answer text and the preset evaluation criteria are filled into the preset template to obtain the text to be evaluated; after the text to be evaluated is delivered to the preset large language model (LLMs), the beam search encoding can be used to return the first preset number of initial evaluation results with the largest generated prediction probability as candidate evaluation results; through the election mechanism, the score threshold is determined according to the evaluation score of each candidate evaluation result, and the candidate evaluation result with an evaluation score greater than the score threshold is used as a reference evaluation result; based on the retained reference evaluation result, a new prompt word is generated as the input of the LLMs model, and the reference evaluation results are integrated by the LLMs model, and multiple reference evaluation results are organically integrated into one to obtain the target evaluation result.
[0107] The text evaluation method proposed in the embodiment of the present invention can be applied to Figure 3 In the text analysis framework shown in Figure 3 As shown, the following processes are performed in the text analysis framework: large language models (LLMs) role setting (set the role); prompt rendering (Prompt rendering), using large language models (LLMs) to raise text analysis questions; using large language models (LLMs) to answer text analysis questions and obtain text analysis answers (Documents retrieval); text analysis answer selection and evaluation (Answer choice / evaluation); based on the multi-path feedback that meets the expectations in the text analysis question raising stage, the text analysis question answering stage, the text analysis answer selection and evaluation stage, multi-layer sample enhancement is performed in the text analysis question raising stage, the text analysis question answering stage, the text analysis answer selection and evaluation stage. The text evaluation method in the embodiment of the present invention can be applied to Figure 3 The text analysis answer evaluation stage.
[0108] (1) Role Setting of Large Language Model
[0109] Set the role of the large language model according to the type of text to be analyzed. Let the large language model know the set role through prompt rendering. For example, using OpenAI, the role of the large language model can be set through the initial system role.
[0110] For example, in engineering implementation, two class instances can be created, each of which uses the call of the large language model as a method that can be used by itself. Each instance has its own Prompt attribute, and the role of the large language model is set as the questioner and the answerer respectively.
[0111] (2) Raising text analysis questions
[0112] For the questioner, the Prompt paradigm is combined with the set role to enable the large language model to independently raise text analysis questions corresponding to the text to be analyzed. The guidance of the Prompt paradigm allows the large language model to think about which text analysis questions should be raised under the current role.
[0113] Optionally, the Prompt paradigm can be a chain thinking method such as COT (Chain of Thoughts); or it can be a more complex SA (Self-Ask) + SE (Search Engine) method, which also shows the thinking process of why this role asks a certain text analysis question, and can also use the search engine to assist in finding professional questions that can be used for text analysis in the professional field related to the role during the thinking process; it can also further use the ReAct (Reasoning and Action) method to further decompose specific text analysis problems, and ask single-hop or even recursive questions to the decomposed questions to enhance the large language model's understanding of text analysis problems; of course, other methods can also be used to analyze the text to be analyzed in order to raise text analysis questions.
[0114] (3) Answer text analysis questions
[0115] As an answerer, you can build a tool library that can be used by large language models, and use your own knowledge, network resources, and text vector libraries to answer text analysis questions based on the question and answer context.
[0116] Define the action space (Action Space) that can be called by the respondent. In addition to the API (Application Program Interface) that can be called by the third party (such as search engine interface, etc.), the reasoning interface of various models can also be registered in the Action Space.
[0117] Guided by the prompt paradigm, the large language model can be used to understand text analysis questions based on the question and answer history of the questioner and the answerer, and choose the way to answer the text analysis questions. The world knowledge accumulated during pre-training can be used to answer; or the interface call can be used to solve the problem based on the understanding of the interface description in the Action Space. For example, for the understanding of the plot, the search engine can be used to obtain relevant information. For the analysis of complex and long texts, the question matching based on the text vector library and the integration and answer of related content can be used to get rid of the dilemma that the large language model cannot quickly and cost-effectively inject a large amount of vertical information and the context window limitation during application.
[0118] By analyzing complex long texts, we can obtain the refined content of the text and build a text vector library based on the refined content of the text. The subsequent large language model can convert text analysis questions into text vector library requests, thereby obtaining answers based on the text vector library.
[0119] (4) Text Analysis Answer Evaluation
[0120] Since the analysis of complex long texts is not a simple logic or mathematical problem, it usually does not have a "standard answer" in the traditional sense, and it focuses more on the analysis process rather than the final result. Therefore, an evaluation mechanism can be established for answering the above questions to obtain the score of the text analysis answer. Among them, the process of evaluating the text analysis answer can be implemented by using the text evaluation method proposed in the embodiment of the present invention.
[0121] (5) Multi-layer sample enhancement
[0122] If the score of the text analysis answer is in line with expectations, the input and output in (2) to (4) above are added as feedback samples to the corresponding links to achieve sample reflux, so that the large language model can achieve better performance.
[0123] As can be seen from the above, in the scheme provided by the present application, the preset large language model can evaluate the answer text in the text to be evaluated based on the preset evaluation criteria and question text in the text to be evaluated, wherein the answer text is obtained by the text analysis model answering the question text based on the target text. It can be understood that if the target text is of high quality, then the text analysis model can extract the information required to answer the question text from the target text more comprehensively, thereby obtaining a high-quality answer text. On the contrary, if the target text is of poor quality, then the text analysis model cannot extract the required information from the target text, and the quality of the answer text obtained will be relatively poor. Therefore, the target evaluation result is not only an evaluation of the answer text, but also reflects the quality of the target text. Moreover, in the evaluation process of the text to be evaluated, it is automatically completed by the preset large language model, which does not rely on manual labor, and can reduce the time and labor cost of manual evaluation of answers. At the same time, the preset large language model is evaluated according to the preset evaluation criteria, and the analysis criteria and analysis dimensions in the evaluation process are consistent, and the target evaluation results obtained have high objectivity, diversity and professionalism.
[0124] The text evaluation method provided in the embodiment of the present application can be executed by a text evaluation device. In the embodiment of the present application, a method in which a text evaluation device performs text evaluation is taken as an example to illustrate the device of the text evaluation method provided in the embodiment of the present application.
[0125] Figure 4is a block diagram of a text evaluation device according to an exemplary embodiment, the device comprising:
[0126] The acquisition module 201 is used to acquire the text to be evaluated; the text to be evaluated includes a question text, an answer text and a preset evaluation standard corresponding to the target text, wherein the answer text is obtained by analyzing the target text based on the question text by a text analysis model, and the preset evaluation standard is the evaluation standard used to evaluate the question text and the answer text corresponding to the target text;
[0127] The analysis module 202 is used to input the text to be evaluated into a preset large language model, analyze the text to be evaluated, and obtain multiple candidate evaluation results; the candidate evaluation results include evaluation contents and corresponding evaluation scores of the question text and its corresponding answer text based on the preset evaluation criteria;
[0128] A screening module 203 is used to use the candidate evaluation results whose evaluation scores meet preset conditions as reference evaluation results;
[0129] The fusion module 204 is used to fuse the reference evaluation results to obtain a target evaluation result.
[0130] In one implementation, the acquisition module 201 is used to:
[0131] Obtain the question text, answer text and preset evaluation criteria corresponding to the target text;
[0132] The question text, the answer text and the preset evaluation criteria are filled into a preset template to obtain a text to be evaluated.
[0133] In one implementation, the acquisition module 201 is used to:
[0134] Obtaining a target text and a preset evaluation standard corresponding to the target text;
[0135] Based on the target text, obtaining a corresponding question text;
[0136] The question text is input into a text analysis model, and the text analysis model analyzes the target text based on the question text to obtain an answer text.
[0137] In one implementation, the analysis module 202 is specifically configured to:
[0138] Obtaining a sample evaluation text and a sample evaluation result, wherein the sample evaluation text includes a sample question, a sample answer, and a sample evaluation standard corresponding to the sample text;
[0139] Filling the sample data and the sample evaluation result into the preset template to obtain a sample template text;
[0140] The text to be evaluated and the example template text are input into a preset large language model, context learning is performed on the text to be evaluated and the example template text, and based on the mapping relationship between the example evaluation text and the example evaluation result, a plurality of candidate evaluation results corresponding to the text to be evaluated are determined.
[0141] In one implementation, the analysis module 202 is used to:
[0142] The text to be evaluated is input into a preset large language model, and the preset large language model analyzes the text to be evaluated to obtain multiple initial evaluation results and corresponding prediction probabilities, and based on the order of the prediction probabilities from large to small, a preset number of candidate evaluation results are determined and output from the initial evaluation results.
[0143] In one implementation, the screening module 203 is used to:
[0144] Determining a score threshold based on the candidate evaluation results and the corresponding evaluation scores;
[0145] The candidate evaluation results whose evaluation scores are greater than the score threshold are used as reference evaluation results.
[0146] In one implementation, the fusion module 204 is used to:
[0147] The reference evaluation result is input into the preset large language model, and the preset large language model performs text fusion on the reference evaluation result to obtain a target evaluation result.
[0148] In one implementation, the preset evaluation criteria include multiple evaluation dimensions, and each of the candidate evaluation results corresponds to the multiple evaluation dimensions.
[0149] As can be seen from the above, in the scheme provided by the present application, the preset large language model can evaluate the answer text in the text to be evaluated based on the preset evaluation criteria and question text in the text to be evaluated, wherein the answer text is obtained by the text analysis model answering the question text based on the target text. It can be understood that if the target text is of high quality, then the text analysis model can extract the information required to answer the question text from the target text more comprehensively, thereby obtaining a high-quality answer text. On the contrary, if the target text is of poor quality, then the text analysis model cannot extract the required information from the target text, and the quality of the answer text obtained will be relatively poor. Therefore, the target evaluation result is not only an evaluation of the answer text, but also reflects the quality of the target text. Moreover, in the evaluation process of the text to be evaluated, it is automatically completed by the preset large language model, which does not rely on manual labor, and can reduce the time and labor cost of manual evaluation of answers. At the same time, the preset large language model is evaluated according to the preset evaluation criteria, and the analysis criteria and analysis dimensions in the evaluation process are consistent, and the target evaluation results obtained have high objectivity, diversity and professionalism.
[0150] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0151] Figure 5 It is a block diagram of an electronic device for text evaluation according to an exemplary embodiment, comprising a processor and a memory, wherein the memory is used to store computer programs; and the processor is used to execute the programs stored in the memory.
[0152] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0153] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0154] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor of an electronic device to perform the above method. Optionally, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0155] In an exemplary embodiment, a computer program product is also provided. When the computer program product is executed on a computer, the computer implements the above-mentioned review text evaluation method.
[0156] As can be seen from the above, the technical solution provided by the embodiment of the embodiment of the present invention, the preset large language model can evaluate the answer text in the text to be evaluated according to the preset evaluation criteria and the question text in the text to be evaluated, wherein the answer text is obtained by the text analysis model answering the question text based on the target text. It can be understood that if the target text is of high quality, then the text analysis model can extract the information required to answer the question text from the target text more comprehensively, thereby obtaining a high-quality answer text. On the contrary, if the target text quality is poor, then the text analysis model cannot extract the required information from the target text, and the quality of the obtained answer text will be relatively poor. Therefore, the target evaluation result is not only an evaluation of the answer text, but also can reflect the quality of the target text. Moreover, in the evaluation process of the text to be evaluated, it is automatically completed by the preset large language model, which does not rely on manual labor, and can reduce the time and labor cost of manual evaluation of answers. At the same time, the preset large language model is evaluated according to the preset evaluation criteria, and the analysis criteria and analysis dimensions in the evaluation process are consistent, and the obtained target evaluation results have high objectivity, diversity and professionalism.
[0157] Figure 6 is a block diagram of a device 800 for text evaluation according to an exemplary embodiment.
[0158] For example, the apparatus 800 may be a mobile phone, a computer, a digital broadcast electronic device, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.
[0159] Reference Figure 6 , the device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .
[0160] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.
[0161] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0162] The power supply component 807 provides power to the various components of the device 800. The power supply component 807 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.
[0163] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data to be processed. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
[0164] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.
[0165] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.
[0166] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800, and the sensor assembly 814 can also detect the position change of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0167] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0168] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to execute the methods described in the first and second aspects.
[0169] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, and the instructions can be executed by the processor 820 of the device 800 to complete the above method. Optionally, for example, the storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0170] In an exemplary embodiment, a computer program product comprising instructions is also provided. When the computer program product is run on a computer, the computer is enabled to execute the text evaluation method described in any one of the above embodiments.
[0171] As can be seen from the above, the technical solution provided by the embodiment of the embodiment of the present invention, the preset large language model can evaluate the answer text in the text to be evaluated according to the preset evaluation criteria and the question text in the text to be evaluated, wherein the answer text is obtained by the text analysis model answering the question text based on the target text. It can be understood that if the target text is of high quality, then the text analysis model can extract the information required to answer the question text from the target text more comprehensively, thereby obtaining a high-quality answer text. On the contrary, if the target text quality is poor, then the text analysis model cannot extract the required information from the target text, and the quality of the obtained answer text will be relatively poor. Therefore, the target evaluation result is not only an evaluation of the answer text, but also can reflect the quality of the target text. Moreover, in the evaluation process of the text to be evaluated, it is automatically completed by the preset large language model, which does not rely on manual labor, and can reduce the time and labor cost of manual evaluation of answers. At the same time, the preset large language model is evaluated according to the preset evaluation criteria, and the analysis criteria and analysis dimensions in the evaluation process are consistent, and the obtained target evaluation results have high objectivity, diversity and professionalism.
[0172] Those skilled in the art will readily appreciate other implementations of the embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The embodiments of the present invention are intended to cover any variations, uses, or adaptations of the embodiments of the present invention that follow the general principles of the embodiments of the present invention and include common knowledge or customary technical means in the art that are not disclosed in the embodiments of the present invention. The specification and embodiments are to be regarded as exemplary only, and the true scope and spirit of the embodiments of the present invention are indicated by the following claims.
[0173] It should be understood that the embodiments of the present invention are not limited to the precise structures described above and shown in the drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the embodiments of the present invention is limited only by the appended claims.
Claims
1. A text evaluation method, characterized in that: include: Obtaining a text to be evaluated; the text to be evaluated includes a question text, an answer text, and a preset evaluation standard corresponding to a target text, wherein the answer text is obtained by analyzing the target text based on the question text by a text analysis model, and the preset evaluation standard is an evaluation standard used to evaluate the question text and the answer text corresponding to the target text; Inputting the text to be evaluated into a preset large language model, analyzing the text to be evaluated, and obtaining a plurality of candidate evaluation results; the candidate evaluation results include evaluation contents and evaluation scores of the question text and its corresponding answer text based on the preset evaluation criteria; The candidate evaluation results whose evaluation scores meet the preset conditions are used as reference evaluation results; The reference evaluation results are fused to obtain the target evaluation results.
2. The text evaluation method according to claim 1, characterized in that The step of obtaining the text to be evaluated includes: Obtain the question text, answer text and preset evaluation criteria corresponding to the target text; The question text, the answer text and the preset evaluation criteria are filled into a preset template to obtain a text to be evaluated.
3. The text evaluation method according to claim 2, characterized in that The step of obtaining the question text, answer text and preset evaluation criteria corresponding to the target text includes: Obtaining a target text and a preset evaluation standard corresponding to the target text; Based on the target text, obtaining a corresponding question text; The question text is input into a text analysis model, and the text analysis model analyzes the target text based on the question text to obtain an answer text.
4. The text evaluation method according to claim 2, characterized in that Before inputting the text to be evaluated into a preset large language model and analyzing the text to be evaluated to obtain a plurality of candidate evaluation results, the method further includes: Obtaining a sample evaluation text and a sample evaluation result, wherein the sample evaluation text includes a sample question, a sample answer, and a sample evaluation standard corresponding to the sample text; Filling the sample data and the sample evaluation result into the preset template to obtain a sample template text; The text to be evaluated is input into a preset large language model, and the text to be evaluated is analyzed to obtain multiple candidate evaluation results, including: The text to be evaluated and the example template text are input into a preset large language model, context learning is performed on the text to be evaluated and the example template text, and based on the mapping relationship between the example evaluation text and the example evaluation result, a plurality of candidate evaluation results corresponding to the text to be evaluated are determined.
5. The text evaluation method according to claim 1, characterized in that The text to be evaluated is input into a preset large language model, and the text to be evaluated is analyzed to obtain multiple candidate evaluation results, including: The text to be evaluated is input into a preset large language model, and the preset large language model analyzes the text to be evaluated to obtain multiple initial evaluation results and corresponding prediction probabilities, and based on the order of the prediction probabilities from large to small, a preset number of candidate evaluation results are determined and output from the initial evaluation results.
6. The text evaluation method according to claim 1, characterized in that: The candidate evaluation results whose evaluation scores meet the preset conditions are used as reference evaluation results, including: Determining a score threshold based on the candidate evaluation results and the corresponding evaluation scores; The candidate evaluation results whose evaluation scores are greater than the score threshold are used as reference evaluation results.
7. The text evaluation method according to claim 1, characterized in that The step of fusing the reference evaluation results to obtain a target evaluation result includes: The reference evaluation result is input into the preset large language model, and the preset large language model performs text fusion on the reference evaluation result to obtain a target evaluation result.
8. The text evaluation method according to claim 1, characterized in that: The preset evaluation criteria include multiple evaluation dimensions, and each of the candidate evaluation results corresponds to the multiple evaluation dimensions.
9. A text evaluation device, characterized in that: include: An acquisition module is used to acquire a text to be evaluated; the text to be evaluated includes a question text, an answer text and a preset evaluation standard corresponding to the target text, wherein the answer text is obtained by analyzing the target text based on the question text by a text analysis model, and the preset evaluation standard is an evaluation standard used to evaluate the question text and the answer text corresponding to the target text; An analysis module, used for inputting the text to be evaluated into a preset large language model, analyzing the text to be evaluated, and obtaining a plurality of candidate evaluation results; the candidate evaluation results include evaluation contents and corresponding evaluation scores of the question text and its corresponding answer text based on the preset evaluation criteria; A screening module is used to select the candidate evaluation results whose evaluation scores meet the preset conditions as reference evaluation results; The fusion module is used to fuse the reference evaluation results to obtain the target evaluation results.
10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the text evaluation method according to any one of claims 1 to 8 are implemented.
11. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the text evaluation method according to any one of claims 1 to 8.
12. A computer program product, characterized in that The program product is stored in a storage medium, and the program product is executed by at least one processor to implement the text evaluation method according to any one of claims 1 to 8.
Citation Information
Patent Citations
General text quality evaluation method based on large language model
CN117634468A
Text summary generation method and apparatus, and computer device and readable storage medium
WO2021217987A1
Cited By
Content evaluation method and device, medium, electronic equipment and program product
CN120431062A
Story question and answer evaluation method and system based on multi-dimensional score and emotion association
CN121168638A
Text processing method, training method of text processing model and electronic equipment
CN121658630A
