Text evaluation method and system, storage medium and processor

Abstract

The invention discloses a text evaluation method and system, a storage medium and a processor, relates to the technical field of text synthesis and evaluation, and solves the problems that in the prior art, an existing evaluation algorithm is relatively single and does not well balance the evaluation rate and precision. The efficiency and accuracy of the evaluation process can be remarkably improved, a large number of texts are rapidly processed through the automatic evaluation model, and meanwhile the high quality of the evaluation result is ensured. According to the scheme, through flexible balance parameter and objective function optimization, the evaluation speed and accuracy are allowed to be balanced, and the requirements of different application scenes are met. In addition, the method also has strong adaptability and expandability, can be easily adjusted to adapt to text evaluation tasks in different fields, and is evolved along with the development of a new technology. By reducing the dependence on manual evaluation, the scheme not only reduces the cost, but also improves the transparency and interpretability of decisions, and provides a powerful and flexible evaluation tool for text generation and processing.

Description

Technical Field

[0001] The present invention relates to the technical field of text synthesis and evaluation, and in particular to a text evaluation method, system, storage medium and processor. Background Art

[0002] In the power industry, grid operators face the challenge of generating accurate and consistent fault reports in real time. To improve efficiency, grid systems have begun to adopt automated text generation technology, which extracts key entities and relationships from structured data, such as "Transformer A failed in region B", and then automatically synthesizes descriptive text based on preset text templates, such as "[Entity] [Event] occurred in [Location]". This process not only speeds up the transmission of information, but also ensures the consistency and accuracy of reports, which is crucial for quickly responding to faults and maintaining grid stability. However, although text synthesis technology plays a key role in improving the efficiency of grid management, it also faces the risk of generating erroneous information, which may have a negative impact on decision making.

[0003] Currently, although there are some algorithms for evaluating the quality of synthetic text, they are often based on single methods, such as rule-based checking or machine learning models, which may perform well in specific fields but may not be accurate enough in other fields. For example, one field may be more suitable for using deep learning-based models to evaluate text because they can capture complex semantic relationships, while in another field, rule-based methods may be more suitable because they can quickly identify format errors. In addition, existing evaluation algorithms often do not balance the rate and accuracy of evaluation well. In the scenario of power grid fault reporting, it is crucial to generate reports quickly, but the evaluation process cannot sacrifice accuracy. Therefore, a more flexible and comprehensive evaluation method is needed that can select the most appropriate evaluation algorithm according to the characteristics and needs of different texts and find the best balance between evaluation speed and accuracy. This comprehensive evaluation method will be able to better adapt to the specific needs of different fields, improve the reliability of synthetic texts, and ensure the efficient and safe operation of power grid systems.

[0004] In view of this, a text evaluation method, system, storage medium and processor are needed. Summary of the invention

[0005] In view of the problem that the existing evaluation algorithms in the prior art are relatively simple and do not balance the evaluation rate and accuracy well, the present invention provides a text evaluation method, system, storage medium and processor, which can select the most appropriate evaluation algorithm according to the characteristics and requirements of different texts and find the best balance between evaluation speed and accuracy. This comprehensive evaluation method will be able to better adapt to the specific needs of different fields, improve the reliability of synthetic texts, and ensure the efficient and safe operation of power grid systems. The specific technical solutions are as follows:

[0006] A text evaluation method comprises the following steps:

[0007] Define several evaluation models, which are used to compare the synthesized text with the reference text and give a similarity score;

[0008] Extract the feature vector of the synthetic text, including statistical features and semantic features, and then send the feature vector of the synthetic text to the evaluation model, and record the evaluation score and calculation time output by each model;

[0009] The objective function is constructed with the goal of minimizing the computation time of the evaluation model and maximizing the accuracy of the evaluation score;

[0010] Solve the objective function and obtain the optimal model combination.

[0011] Preferably, the evaluation model includes a rule-based evaluation model, and the implementation steps are as follows:

[0012] The text is evaluated using a set of predefined rules, including grammar checking and spelling checking.

[0013] Check whether the text conforms to the preset structure and format. For each rule, if the text passes the check, a positive score is assigned, otherwise a negative score is assigned. The total evaluation score is the sum of all rule scores.

[0014] Preferably, the evaluation model includes a machine learning model, and the steps are as follows:

[0015] Use machine learning algorithms to evaluate text based on a set of features, including TF-IDF and vocabulary richness.

[0016] Train a classifier to classify text into two categories: "appropriate" and "inappropriate".

[0017] The score is based on the confidence of the classifier, i.e. the probability of the correct classification.

[0018] The computation time includes the time for feature extraction and model prediction.

[0019] Preferably, the evaluation model includes a deep learning model, and the steps are as follows:

[0020] Use the pre-trained BERT model and fine-tune it to adapt to domain-specific text evaluation tasks;

[0021] Using the output layer of the BERT model, a probability score representing the suitability of the text is obtained through the softmax function;

[0022] The computation time includes the time to load the model, extract features, and perform model inference.

[0023] Preferably, the method for semantic similarity comprises the following steps:

[0024] Use Siamese networks or other deep learning models to compare the semantic similarity between generated text and reference text;

[0025] By calculating the cosine similarity between the generated text and the reference text, a score between 0 and 1 is obtained. The higher the score, the closer the semantics of the text is to the reference text.

[0026] The computation time includes the time for text vectorization and similarity calculation.

[0027] Preferably, the objective function is as follows:

[0028] J = α·T+(1-α)·(1-Acc)

[0029] Where T is the calculation time of the model, Acc is the evaluation accuracy of the model, and α is a balance parameter with a value between 0 and 1.

[0030] A text evaluation system, applied to the method as described above, comprising:

[0031] An evaluation unit, wherein a plurality of evaluation models are provided in the evaluation unit, and the evaluation models are used to compare the synthesized text with the reference text and give a similarity score;

[0032] The feature extraction unit extracts the feature vector of the synthetic text, including statistical features and semantic features, and then sends the feature vector of the synthetic text to the evaluation model, and records the evaluation score and calculation time output by each model;

[0033] An objective function construction unit constructs an objective function with the goal of minimizing the computation time of the evaluation model and maximizing the accuracy of the evaluation score;

[0034] Solve the unit, solve the objective function, and obtain the optimal model combination.

[0035] The terminal is connected to the solving unit for receiving the optimal model combination output by the solving unit.

[0036] Preferably, it also includes a constraint condition construction unit, which is used to construct the constraint conditions of the objective function, specifically as follows:

[0037] Accuracy constraints:

[0038] |Acc i -Ref|≤θ;

[0039] Where Acc i is the evaluation score of model i, Ref is the reference score, and θ is the maximum error accepted;

[0040] Compute time constraints:

[0041] T i ≤T max ;

[0042] Among them, T i is the computation time of model i, T max is the maximum allowed computation time.

[0043] Resource usage constraints:

[0044] R i ≤R max ;

[0045] Among them, R i is the resource usage of model i, R max is the maximum allowed resource usage.

[0046] A computer-readable storage medium includes a stored program, wherein when the program is executed, a device where the computer-readable storage medium is located is controlled to execute the text evaluation method as described above.

[0047] A processor is used to run a program, wherein the program executes the text evaluation method as described above when it is run.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] The present invention can significantly improve the efficiency and accuracy of the evaluation process, quickly process large amounts of text through an automated evaluation model, and ensure the high quality of the evaluation results. The scheme allows a trade-off between evaluation speed and accuracy through flexible balance parameters and objective function optimization to meet the needs of different application scenarios. In addition, it has strong adaptability and scalability, can be easily adjusted to adapt to text evaluation tasks in different fields, and evolve with the development of new technologies. By reducing reliance on manual evaluation, the scheme not only reduces costs, but also improves the transparency and interpretability of decisions, providing a powerful and flexible evaluation tool for text generation and processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0051] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0054] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0055] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0056] In one embodiment of the present invention, a text evaluation method is provided. Figure 1 As shown, the following steps are included:

[0057] Step 1: Define several evaluation models, which are used to compare the synthesized text with the reference text and give a similarity score;

[0058] To provide a concrete solution, we will explain four evaluation models in detail: Model A, B, C, D. The evaluation method, output evaluation score, and computation time of each model are calculated as follows:

[0059] Model A: Rule-based approach, the steps are as follows:

[0060] Evaluate text using a set of predefined rules (e.g. grammar check, spell check).

[0061] Checks whether the text conforms to the preset structure and format. For each rule, a positive score is assigned if the text passes the check, otherwise a negative score is assigned. The total evaluation score is the sum of all rule scores.

[0062] Due to the rule-based approach, the computation time is fixed.

[0063] Model B: Machine learning approach, the steps are as follows:

[0064] Use a support vector machine (SVM) or other machine learning algorithm to evaluate text based on a set of features (e.g. TF-IDF, vocabulary richness).

[0065] Train a classifier to classify text into two categories: "appropriate" and "inappropriate".

[0066] The score is based on the confidence of the classifier, i.e. the probability of the correct classification.

[0067] The computation time includes the time for feature extraction and model prediction.

[0068] Model C: Deep learning method, the steps are as follows:

[0069] Use the pre-trained BERT model and fine-tune it to suit domain-specific text evaluation tasks.

[0070] Using the output layer of the BERT model, a probability score representing the suitability of the text is obtained through the softmax function.

[0071] The computation time includes the time to load the model, extract features, and perform model inference.

[0072] Model D: Semantic similarity-based approach, the steps are as follows:

[0073] Use Siamese networks or other deep learning models to compare the semantic similarity between the generated text and the reference text.

[0074] By calculating the cosine similarity between the generated text and the reference text, a score between 0 and 1 is obtained. The higher the score, the closer the semantics of the text is to the reference text.

[0075] The computation time includes the time for text vectorization and similarity calculation.

[0076] Among them, the commonly used evaluation indicators of semantic similarity include:

[0077] (1) Inner product distance

[0078] The inner product distance calculates the difference in direction between two vectors. The smaller the angle, the more similar they are. Therefore, the larger the inner product value, the more similar they are. In a geometric sense, the inner product calculates the length of the perpendicular projection of one vector on another vector.

[0079] The calculation formula of the inner product distance is:

[0080]

[0081] (2) Euclidean distance

[0082] Euclidean distance calculates the shortest straight-line distance between two points. The smaller the distance value, the more similar they are. Euclidean distance is one of the most commonly used distance calculation methods. It is widely used and suitable for scenarios with complete data and unified data dimensions. Euclidean distance can reflect the absolute difference of individual numerical characteristics, so it is more used in analysis that needs to reflect the difference from the numerical size of the dimension, such as using user behavior indicators to analyze the similarity or difference of user value.

[0083] The calculation formula of Euclidean distance is:

[0084]

[0085] (3) Cosine distance

[0086] The cosine distance calculates the cosine value of the angle between two vectors. The smaller the angle, the more similar they are. Therefore, the larger the cosine similarity value, the more similar they are. Cosine distance and inner product distance are more about distinguishing differences in direction, and are not sensitive to absolute values. They are more used to distinguish similarities and differences in interests by using users' ratings of content, and at the same time correct the problem of inconsistent metrics among users.

[0087] The calculation formula for cosine distance is:

[0088]

[0089] Assuming that the normalized vectors A and B are A' and B' respectively, then:

[0090]

[0091] (4) Hamming distance

[0092] The Hamming distance calculates the distance between binary strings. The Hamming distance between two strings of equal length is defined as the minimum number of substitutions required to transform one string into the other.

[0093] (5) Jaccard distance

[0094] The Jaccard similarity coefficient calculates the similarity between data sets by the ratio of the number of intersections and the number of unions of the data sets. The calculation formula can be expressed as:

[0095]

[0096] Jaccard distance is an indicator used to measure the difference between two data sets and is defined as 1 minus the Jaccard similarity coefficient. Jaccard distance is suitable for string similarity measurement.

[0097] The calculation formula of Jaccard distance is:

[0098]

[0099] Of course, in addition to the ABCD model, other evaluation models can also be used. This step aims to use some existing evaluation models to evaluate the similarity between the synthesized text and the reference text. It only needs to obtain a similarity score and calculation time. In addition to using some existing methods such as similarity calculation or training neural network models, an expert scoring step can also be added as an additional model. The score and calculation time that reflect the similarity can also be obtained. Later, it can also be put into the objective function for optimization and solution. The above-mentioned synthesized text is a synthesized text obtained based on feature extraction such as entity-relationship extraction and pre-established synthesis rules or other synthesis means. The reference text is a standard text used to test whether the synthesized text is qualified, which can be directly provided by the staff.

[0100] Step 2: Extract feature vectors of synthetic text, including statistical features and semantic features;

[0101] In the text evaluation scheme, feature extraction is one of the key steps, which involves extracting useful information from the synthetic text for use by the evaluation model. The following introduces some common statistical features and semantic features, and their role in text evaluation:

[0102] Statistical features include:

[0103] Term Frequency (TF): Calculates the number of occurrences of each word in a text. It is used to reflect the importance of a word in a text.

[0104] Sentence Length: Calculates the average length of sentences in your text. Sentence length can affect the readability and flow of your text.

[0105] Lexical Richness: Calculates the number of different words or type-token ratio in a text. It is used to measure the diversity of the text and the richness of expression.

[0106] Sentence complexity: Counts the number of clauses in a sentence or the complexity of the sentence structure. Used to assess the grammatical complexity of a text.

[0107] Text length: Count the total number of words or characters in the text. Text length can affect the depth and detail of the information.

[0108] Punctuation usage: Analyze the frequency and diversity of punctuation usage. Appropriate punctuation usage can improve the readability of the text.

[0109] The semantic features are as follows:

[0110] TF-IDF (Term Frequency-Inverse Document Frequency): Calculates the frequency of a word in a text while taking into account the distribution of the word in the corpus. It is used to reflect the importance of a word in a text while suppressing the influence of common words.

[0111] Word Embeddings: Convert words into dense vector representations, such as Word2Vec or GloVe, to capture the semantic information and contextual relationships of words.

[0112] Sentence Embeddings: Convert the entire sentence into a vector representation, such as using BERT or Sent2Vec. Used to capture the semantic information of the entire sentence.

[0113] Topic Modeling: Use models such as LDA (Latent Dirichlet Allocation) to identify topics in text. Used to understand the main content and structure of the text.

[0114] Semantic Role Labeling (SRL): Analyze the semantic roles of words in a sentence, such as agent, patient, etc. It is used to deeply understand the semantic structure of a sentence.

[0115] Dependency Relations: Analyze the dependency relationships between words in a sentence. Used to capture the grammatical structure and semantic connections of a sentence.

[0116] After extracting these features, they can be used to:

[0117] Train evaluation model: Use features as input to train a machine learning or deep learning model to predict the quality of the text and then derive a rating.

[0118] Similarity calculation: used to calculate the similarity between the generated text and the reference text, and then derive a score.

[0119] Step 3: Send the feature vector of the synthesized text into the evaluation model and record the evaluation score and calculation time output by each model;

[0120] This step aims to obtain the evaluation score and calculation time. For details, please refer to the evaluation scores and calculation time of several models listed in step 1. In order to unify the subsequent calculations, the evaluation scores of each model are normalized.

[0121] Step 4: Construct an objective function with the goal of minimizing the computation time of the evaluation model and maximizing the accuracy of the evaluation score;

[0122] The objective function is as follows:

[0123] J = α·T+(1-α)·(1-Acc)

[0124] Where T is the calculation time of the model, Acc is the evaluation accuracy of the model, and α is a balance parameter with a value between 0 and 1.

[0125] The main purpose of introducing the balance parameter α in the objective function is to weigh the importance of different evaluation factors. These factors are the calculation rate (time) of the evaluation model and the accuracy of the evaluation score. The balance parameter allows us to adjust the relative weights of these two factors according to different situations during the optimization process.

[0126] In this way, the objective function allows the user or system to adjust the trade-off between evaluation accuracy and computation rate according to specific needs. For example, in some cases, the accuracy of the evaluation may be more important, while in other cases, the speed of the evaluation may be more important. It enables the model to adapt to different application scenarios. In time-sensitive applications, faster evaluation speed may be required; while in scenarios with higher accuracy requirements, you may be willing to sacrifice some speed in exchange for higher evaluation quality. Moreover, in multi-objective optimization problems, there is usually no single solution that can optimally meet all objectives at the same time. The balance parameter helps us find the "Pareto front" of optimal solutions, that is, a set of solutions that provide the best trade-off between different objectives. In addition, the existence of the balance parameter allows the system to dynamically adjust the parameters based on real-time feedback or performance indicators. For example, if the system detects that computing resources are tight, it can increase the emphasis on computing rate, and vice versa.

[0127] In addition, some constraints can be added to make the solution more reasonable. The constraints are set as follows:

[0128] Taking the automatic generation of power grid fault reports as an example, specific data and operation steps are given.

[0129] (1) Accuracy constraints:

[0130] The evaluation score of each model should be close to the reference score (the score given by human experts). The details are as follows:

[0131] |Acc i -Ref|≤θ;

[0132] Where Acc i is the evaluation score of model i, Ref is the reference score, and θ is the maximum error accepted;

[0133] (2) Calculation time constraints:

[0134] The computation time of the model should be within the predetermined threshold. Specifically:

[0135] T i ≤T max ;

[0136] Among them, T i is the computation time of model i, T max is the maximum allowed computation time.

[0137] (3) Resource usage constraints:

[0138] The resource usage of the model (such as CPU, memory) should be within the range allowed by the system. The details are as follows:

[0139] R i ≤R max ;

[0140] Among them, R i is the resource usage of model i, R max is the maximum allowed resource usage.

[0141] Step 5: Solve the objective function and obtain the optimal model combination. That is, use an optimization algorithm (such as genetic algorithm, dynamic programming) to solve the minimum value of the objective function. The details are as follows:

[0142] Define a loss function, such as a combination of weighted evaluation time and evaluation score.

[0143] The model combination is adjusted through the optimization algorithm until the model combination that minimizes the loss function is found.

[0144] After step five, an optimal model combination can be obtained. At this time, this optimal model combination can be used to process the same type of synthetic text. This optimal model is the optimal processing method for the synthetic text input in step one.

[0145] In one embodiment of the present invention, a text evaluation system is further provided, which is applied to the above method and includes:

[0146] An evaluation unit, wherein a plurality of evaluation models are provided in the evaluation unit, and the evaluation models are used to compare the synthesized text with the reference text and give a similarity score;

[0147] The feature extraction unit extracts the feature vector of the synthetic text, including statistical features and semantic features, and then sends the feature vector of the synthetic text to the evaluation model, and records the evaluation score and calculation time output by each model;

[0148] An objective function construction unit constructs an objective function with the goal of minimizing the computation time of the evaluation model and maximizing the accuracy of the evaluation score;

[0149] Solve the unit, solve the objective function, and obtain the optimal model combination.

[0150] The terminal is connected to the solving unit for receiving the optimal model combination output by the solving unit.

[0151] In summary, the implementation of this comprehensive text evaluation scheme can significantly improve the efficiency and accuracy of the evaluation process, quickly process large amounts of text through automated evaluation models, and ensure the high quality of the evaluation results. This scheme allows for a trade-off between evaluation speed and accuracy through flexible balance parameters and objective function optimization to meet the needs of different application scenarios. In addition, it has strong adaptability and scalability, can be easily adjusted to adapt to text evaluation tasks in different fields, and evolve with the development of new technologies. By reducing reliance on manual evaluation, this scheme not only reduces costs, but also improves the transparency and explainability of decisions, providing a powerful and flexible evaluation tool for text generation and processing.

[0152] Those of ordinary skill in the art will appreciate that the units of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition of each example has been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0153] In the embodiments provided by the present invention, it should be understood that the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units can be combined into one unit, one unit can be split into multiple units, or some features can be ignored, etc.

[0154] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0155] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-0nlyMemory), random access memory (RAM, RandomAccessMemory), mobile hard disk, magnetic disk or optical disk, etc., which can store program code.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A text evaluation method, characterized in that: The following steps are involved: Define several evaluation models, which are used to compare the synthesized text with the reference text and give a similarity score; Extract the feature vector of the synthetic text, including statistical features and semantic features, and then send the feature vector of the synthetic text to the evaluation model, and record the evaluation score and calculation time output by each model; The objective function is constructed with the goal of minimizing the computation time of the evaluation model and maximizing the accuracy of the evaluation score; Solve the objective function and obtain the optimal model combination.

2. A text evaluation method according to claim 1, characterized in that: The evaluation model includes a rule-based evaluation model, and the implementation steps are as follows: evaluating the text using a set of predefined rules, the predefined rules including grammar checking and spelling checking; Check whether the text conforms to the preset structure and format. For each rule, if the text passes the check, a positive score is assigned, otherwise a negative score is assigned. The total evaluation score is the sum of all rule scores.

3. A text evaluation method according to claim 1, characterized in that: The evaluation model includes the machine learning model, and the steps are as follows: Use machine learning algorithms to evaluate text based on a set of features, including TF-IDF and vocabulary richness; Train a classifier to classify text into two categories: "appropriate" and "inappropriate"; The score is based on the confidence of the classifier, i.e. the probability of correct classification; The computation time includes the time for feature extraction and model prediction.

4. A text evaluation method according to claim 1, characterized in that: The evaluation model includes a deep learning model, and the steps are as follows: Use the pre-trained BERT model and fine-tune it to adapt to domain-specific text evaluation tasks; Using the output layer of the BERT model, a probability score representing the suitability of the text is obtained through the softmax function; The computation time includes the time to load the model, extract features, and perform model inference.

5. A text evaluation method according to claim 1, characterized in that: The method for semantic similarity is as follows: Use Siamese networks or other deep learning models to compare the semantic similarity between generated text and reference text; By calculating the cosine similarity between the generated text and the reference text, a score between 0 and 1 is obtained. The higher the score, the closer the semantics of the text is to the reference text. The computation time includes the time for text vectorization and similarity calculation.

6. A text evaluation method according to claim 1, characterized in that: The objective function is as follows: J = α·T+(1-α)·(1-Acc) Where T is the calculation time of the model, Acc is the evaluation accuracy of the model, and α is a balance parameter with a value between 0 and 1.

7. A text evaluation system, characterized in that: The method applied to any one of claims 1 to 6 comprises: An evaluation unit, wherein a plurality of evaluation models are provided in the evaluation unit, and the evaluation models are used to compare the synthesized text with the reference text and give a similarity score; The feature extraction unit extracts the feature vector of the synthetic text, including statistical features and semantic features, and then sends the feature vector of the synthetic text to the evaluation model, and records the evaluation score and calculation time output by each model; An objective function construction unit constructs an objective function with the goal of minimizing the computation time of the evaluation model and maximizing the accuracy of the evaluation score; Solving unit, solving the objective function and obtaining the optimal model combination; The terminal is connected to the solving unit for receiving the optimal model combination output by the solving unit.

8. A text evaluation system according to claim 7, characterized in that: It also includes a constraint condition construction unit, which is used to construct the constraint conditions of the objective function, as follows: Accuracy constraints: Acc i -Ref≤θ; Where Acc i is the evaluation score of model i, Ref is the reference score, and θ is the maximum error accepted; Compute time constraints: T i ≤T max ; Among them, T i is the computation time of model i, T max is the maximum allowed computation time; Resource usage constraints: R i ≤R max ; Among them, R i is the resource usage of model i, R max is the maximum allowed resource usage.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the text evaluation method according to any one of claims 1 to 6.

10. A processor, characterized in that: The processor is used to run a program, wherein the program executes the text evaluation method according to any one of claims 1 to 6 when running.

Images