Text content evaluation method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application provide a text content evaluation method and device, electronic equipment and storage medium, the method comprising: obtaining a text to be evaluated; inputting the text to be evaluated into a fusion evaluation model to obtain evaluation results of at least two text evaluation dimensions, wherein the fusion evaluation model is obtained by iteratively training an initial fusion evaluation model based on a plurality of first text samples without annotation information and at least two target evaluation results corresponding to each first text sample, and the at least two target evaluation results are obtained by inputting each first text sample into at least two content evaluation models. The text content evaluation method, device, electronic equipment and storage medium provided by the embodiments of the present application can improve the accuracy of the fusion evaluation model, and thus the accuracy of the text evaluation result can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of text processing, and in particular to a text content evaluation method and device, an electronic device, and a storage medium. BACKGROUND

[0002] In some application fields, it is necessary to evaluate text. For example, in the field of education, it is often necessary to score students' compositions. When correcting students' compositions, evaluators may focus on multiple text evaluation dimensions, such as finding spelling errors and grammatical errors in compositions, identifying highlight sentence patterns, and related content such as detailed description techniques.

[0003] To achieve the above purpose, in the prior art, the training data corresponding to each text evaluation task is usually randomly shuffled and input into a fusion evaluation model to train a model with multi-task text evaluation capability, thereby achieving the purpose of automatically evaluating text, wherein different text evaluation tasks correspond to different evaluation dimensions.

[0004] However, in the above text evaluation process, the sample size of the training data of different text evaluation tasks is often not the same, which will cause the fusion evaluation model to pay too much attention to the task with a large sample size of training data and ignore the task with a small sample size of training data, thereby reducing the accuracy of the fusion evaluation model and making the text evaluation result less accurate. SUMMARY

[0005] The present application provides a text content evaluation method, device, electronic device and storage medium to solve the problem of low accuracy of the fusion evaluation model caused by inconsistent sample size of training data of different tasks in the prior art, which makes the text evaluation result less accurate. The present application improves the accuracy of the fusion evaluation model and improves the accuracy of the text evaluation result.

[0006] The present application provides a text content evaluation method, comprising:

[0007] obtaining a text to be evaluated;

[0008] inputting the text to be evaluated into a fusion evaluation model to obtain evaluation results of at least two text evaluation dimensions, wherein the fusion evaluation model is obtained by iteratively training an initial fusion evaluation model based on a plurality of first text samples without annotation information and at least two target evaluation results corresponding to each first text sample, and the at least two target evaluation results are obtained by inputting each first text sample into at least two content evaluation models.

[0009] The application provides a text content evaluation method, which comprises the following steps:

[0010] The application provides a text content evaluation method, which comprises the following steps:

[0011] The application provides a text content evaluation method, which comprises the following steps:

[0012] The application provides a text content evaluation method, which comprises the following steps:

[0013] The application provides a text content evaluation method, which comprises the following steps:

[0014] The application provides a text content evaluation method, which comprises the following steps:

[0015] The application provides a text content evaluation method, which comprises the following steps:

[0016] The application provides a text content evaluation method, which comprises the following steps:

[0017] The application provides a text content evaluation method, which comprises the following steps:

[0018] The application provides a text content evaluation method, which comprises the following steps:

[0019] The application provides a text content evaluation method, which comprises the following steps:

[0020] The at least two target evaluation results corresponding to the first text sample are obtained based on the following manner:

[0021] For each type of content evaluation model, the first text sample is input into at least two target content evaluation models corresponding to the content evaluation model, to obtain at least two initial evaluation results corresponding to the first text sample.

[0022] Based on the at least two initial evaluation results corresponding to the first text sample, the target evaluation result corresponding to the first text sample is determined.

[0023] According to the evaluation method of the text content provided by the application, the number of the target content evaluation model is at least three;

[0024] The target evaluation result corresponding to the first text sample is determined based on the at least two initial evaluation results corresponding to the first text sample, comprising:

[0025] The initial evaluation result with the highest proportion in the at least three initial evaluation results corresponding to the first text sample is determined as the target evaluation result corresponding to the first text sample.

[0026] According to the evaluation method of the text content provided by the application, the method further comprises:

[0027] Obtain a plurality of second text samples with labeled information, and the labeled information of each second text sample is obtained by evaluating the second text sample based on a target text evaluation dimension, and the target text evaluation dimension is any one of the at least two text evaluation dimensions;

[0028] Based on each second text sample and the corresponding labeled information, an initial content evaluation model is iteratively trained to obtain a content evaluation model corresponding to the target text evaluation dimension.

[0029] The application also provides an evaluation device for text content, comprising:

[0030] The acquisition module is configured to acquire a text to be evaluated.

[0031] The input module is configured to input the text to be evaluated into a fusion evaluation model to obtain evaluation results of at least two text evaluation dimensions, wherein the fusion evaluation model is obtained by iteratively training an initial fusion evaluation model based on a plurality of first text samples without labeled information and at least two target evaluation results corresponding to each first text sample, and the at least two target evaluation results are obtained by inputting each first text sample into at least two types of content evaluation models.

[0032] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the text content evaluation method according to any one of the preceding embodiments when executing the program.

[0033] The application further provides an electronic device comprising a display screen, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a fusion evaluation model in the computer program to process a text to be evaluated and obtain evaluation results of at least two text evaluation dimensions.

[0034] The display screen is configured to display the evaluation results.

[0035] The fusion evaluation model is obtained by iteratively training an initial fusion evaluation model based on a plurality of first text samples without annotation information and at least two target evaluation results corresponding to each of the first text samples, wherein the at least two target evaluation results are obtained by inputting each of the first text samples into at least two content evaluation models.

[0036] The application further provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the text content evaluation method according to any one of the preceding embodiments.

[0037] The application further provides a computer program product comprising a computer program, wherein the computer program is executable by a processor to implement the text content evaluation method according to any one of the preceding embodiments.

[0038] The text content evaluation method, device, electronic device, and storage medium provided by the application can use the target evaluation results of at least two text evaluation dimensions corresponding to the same first text sample as labels to iteratively train an initial fusion evaluation model, so that the difference between the number of first text samples corresponding to different text evaluation dimensions is small, thereby avoiding the problem of inaccurate fusion evaluation model caused by a large difference in the amount of training data samples of text evaluation tasks corresponding to different text evaluation dimensions when training the fusion evaluation model, improving the accuracy of the fusion evaluation model, and further obtaining more accurate text evaluation results after inputting the text to be evaluated into the more accurate fusion evaluation model. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0040] Figure 1 This is a schematic diagram illustrating how existing technologies evaluate text from multiple text evaluation dimensions.

[0041] Figure 2 This is one of the flowcharts illustrating the text content evaluation method provided in this embodiment of the invention;

[0042] Figure 3 This is one of the schematic diagrams illustrating the generation of target evaluation results provided in this embodiment of the invention;

[0043] Figure 4 This is a schematic diagram of sequence annotation of the first text sample provided in an embodiment of the present invention;

[0044] Figure 5 This is a second flowchart illustrating the text content evaluation method provided in this embodiment of the invention.

[0045] Figure 6 This is the second schematic diagram illustrating the generation of target evaluation results provided in this embodiment of the invention;

[0046] Figure 7 This is a schematic diagram illustrating the training of the content evaluation model provided in an embodiment of the present invention;

[0047] Figure 8 This is a schematic diagram illustrating the training of the target content evaluation model provided in an embodiment of the present invention;

[0048] Figure 9 This is a schematic diagram of the structure of the text content evaluation device provided in an embodiment of the present invention;

[0049] Figure 10 This is one of the structural schematic diagrams of the electronic device provided by the present invention;

[0050] Figure 11 This is the second schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0051] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0052] At present, the most widely used in text evaluation is artificial evaluation, that is, the evaluation personnel directly evaluate the text content. However, in some cases, the number of texts to be evaluated is often large, such as teachers need to score hundreds of students' compositions, which will result in high labor cost and time cost of artificial evaluation. Therefore, in the prior art, different text evaluation models are usually trained from multiple text evaluation dimensions to complete the automatic evaluation of the text. Among them, the multiple text evaluation dimensions may include, for example: correcting spelling errors or grammatical errors in the text, identifying complex sentence patterns such as relative clauses, adverbial clauses, appositive clauses in the text, identifying sentences with detailed descriptions such as psychological description, action description or character description in the text.

[0053] Among them, for the spelling and grammar correction task, the goal is to find the position of the spelling error or grammar error in the text and correct the error. This task can be considered as a sequence labeling task; the identification task of complex sentence patterns and sentences with detailed descriptions can be considered as a sentence classification task.

[0054] Specifically, the above three types of text evaluation tasks can be implemented in the manner as shown in Figure 1 Among them, Figure 1 The existing technology for evaluating text from multiple text evaluation dimensions is shown in Figure 1As shown, for the first spelling and grammar correction task, the prior art adopts the GECToR (Grammatical Error Correction: Tag, Not Rewrite) model, which defines 5000 error correction tags, most of which are directly output as tags after correcting the word, such as the tag for modifying "hapy" to "happy" is "$REPLACE_happy". In addition, some special tags classify some errors, such as the tags for modifying "book" to "books" and "friend" to "friends" are both "$TRANSFORM_AGREEMENT_PLURAL", which represents the error correction method of modifying the singular noun to the plural. In terms of model structure, GECToR can be various pre-training language models such as BERT (Bidirectional Encoder Representation from Transformers), RoBERTa (Robustly Optimized BERT Pretraining Approach) and XLNET, etc. Then, after obtaining the hidden layer representation of each word through the above pre-training model, a fully connected layer is connected to output the predicted label of the word.

[0055] In addition, since the last two tasks belong to the sentence classification task, the usual method is to use a pre-training model such as BERT to encode the text, and then take the hidden layer representation of the [CLS] position and connect a fully connected layer to output the predicted label of the position.

[0056] Through the above different tasks, only a single text evaluation dimension can be evaluated, which makes the text evaluation accuracy low and cannot be applied to the above composition correction scene. Therefore, the prior art proposes a model fusion method, that is, the training data corresponding to each text evaluation task is randomly shuffled and input into the model to train a model that can handle multiple text evaluation tasks at the same time, thereby achieving the purpose of text evaluation from multiple text evaluation dimensions.

[0057] However, during the training of the above model, the sample size of different text evaluation tasks is usually different, which leads to the trained model often paying too much attention to the task with more sample size and ignoring the task with less sample size, thereby making the model have poor evaluation effect on the task with less sample size, that is, the text evaluation is not accurate enough. In addition, the above method is trained by labeled samples, and since a large number of samples need to be labeled, the efficiency of model training is low.

[0058] Based on this, the embodiment of the present application proposes a text content evaluation method. The fusion evaluation model in the method is obtained by iteratively training an initial fusion evaluation model based on a plurality of first text samples without annotation information and at least two target evaluation results corresponding to each of the first text samples. The at least two target evaluation results are obtained by inputting each of the first text samples into a plurality of content evaluation models. In this way, in the process of iteratively training the initial fusion evaluation model based on the first text samples and the at least two target evaluation results corresponding to the first text samples, the target evaluation results of at least two text evaluation dimensions corresponding to the same first text sample are used as labels to iteratively train the initial fusion evaluation model. In this way, the difference between the number of first text samples corresponding to different text evaluation dimensions can be small. Thus, the problem of inaccurate fusion evaluation model caused by a large difference in the amount of training data samples of text evaluation tasks corresponding to different text evaluation dimensions when training the fusion evaluation model is avoided. The accuracy of the fusion evaluation model is improved. Furthermore, the obtained text to be evaluated can be input into the more accurate fusion evaluation model to obtain more accurate text evaluation results.

[0059] The following will be described in combination with Figures 2-8 The text content evaluation method provided by the embodiment of the present application can be applied in a text evaluation scene, especially in a scene of correcting student compositions or articles. The text can be Chinese, English, or a text in other languages. The subject executing the method can be a mobile phone, a computer, an electronic device, or any other device capable of model training.

[0060] Figure 2 As shown in FIG. 1, the text content evaluation method provided by the embodiment of the present application includes the following steps. Figure 2

[0061] Step 201: Obtain a text to be evaluated.

[0062] The text to be evaluated can be Chinese, English, or a text in other languages. The specific form of the text to be evaluated is not limited in the embodiment of the present application.

[0063] Specifically, the text to be evaluated can be obtained by a network crawler, a text scanning device, or a camera.

[0064] Step 202: Input the text to be evaluated into a fusion evaluation model to obtain evaluation results of at least two text evaluation dimensions.

[0065] ​The fusion evaluation model is obtained by iteratively training an initial fusion evaluation model based on a plurality of first text samples without annotation information and at least two target evaluation results corresponding to each of the first text samples.

[0066] Specifically, different text evaluation dimensions can be understood as different evaluations of the first text sample under different evaluation criteria, such as whether there is a spelling error, whether there is a complex sentence, and whether there is a detailed description.

[0067] An exemplary target evaluation result generation diagram provided by an embodiment of the present application is shown in FIG. 1. Figure 3 An exemplary target evaluation result generation diagram provided by an embodiment of the present application is shown in FIG. 1. Figure 3 As shown in FIG. 1, the first text sample with no annotation information obtained above is input into n content evaluation models, such as task 1 content evaluation model, task 2 content evaluation model, and task n content evaluation model in FIG. 1. Figure 3 As shown in FIG. 1, the first text sample with no annotation information obtained above is input into n content evaluation models, such as task 1 content evaluation model, task 2 content evaluation model, and task n content evaluation model in FIG. 1.

[0068] After each first text sample is input into the plurality of content evaluation models in FIG. 1, n different target evaluation results corresponding to each first text sample can be obtained, such as task 1 target evaluation result, task 2 target evaluation result, and task n target evaluation result. Figure 3 After each first text sample is input into the plurality of content evaluation models in FIG. 1, n different target evaluation results corresponding to each first text sample can be obtained, such as task 1 target evaluation result, task 2 target evaluation result, and task n target evaluation result.

[0069] Further, after obtaining at least two target evaluation results corresponding to each first text sample, an initial fusion evaluation model is iteratively trained based on the first text sample and the at least two target evaluation results corresponding to each first text sample, to obtain a fusion evaluation model, which is used to evaluate a text to be evaluated from at least two text evaluation dimensions.

[0070] The initial fusion evaluation model can be a pre-trained existing fusion evaluation model or a fusion evaluation model without training.

[0071] Specifically, after obtaining at least two target evaluation results corresponding to each first text sample, all first text samples can be used as training data of an initial fusion evaluation model, at least two target evaluation results corresponding to each first text sample can be merged as label information, the initial fusion evaluation model can be supervised learning, the initial fusion evaluation model can be updated by iterative training until a convergence condition is met, and the final initial fusion evaluation model can be used as a fusion evaluation model.

[0072] It is worth noting that merging at least two target evaluation results corresponding to each first text sample can be understood as superimposing at least two target evaluation results. For example, for the first text sample "I am happy that you come," its corresponding target evaluation results include: there is a spelling error, it is an object clause, and it is a psychological description. These three target evaluation results can be superimposed as the label for "I am happy that you come." Since at least two target evaluation results are obtained based on at least two text evaluation dimensions, the difference in the number of first text samples corresponding to different text evaluation dimensions is less than the preset value, that is, the difference in the number of samples is not significant.

[0073] The fusion evaluation model in the text content evaluation method provided in this embodiment of the invention is obtained by iteratively training an initial fusion evaluation model based on multiple unlabeled first text samples and at least two target evaluation results corresponding to each first text sample. The at least two target evaluation results are obtained by inputting each first text sample into at least two types of content evaluation models. Therefore, during the iterative training of the initial fusion evaluation model based on the first text sample and at least two target evaluation results corresponding to the first text sample, the target evaluation results of at least two text evaluation dimensions corresponding to the same first text sample can be used as labels for iterative training of the initial fusion evaluation model. This minimizes the difference in the number of first text samples corresponding to different text evaluation dimensions, thereby avoiding the problem of inaccurate fusion evaluation models caused by large differences in the training data sample size of text evaluation tasks corresponding to different text evaluation dimensions during training. This improves the accuracy of the fusion evaluation model, allowing the acquired text to be evaluated to be input into a more accurate fusion evaluation model, resulting in more precise text evaluation results.

[0074] In one possible implementation, when inputting the text to be evaluated into the fusion evaluation model to obtain evaluation results for at least two text evaluation dimensions, the following approach can be taken: input the text to be evaluated into the fusion evaluation model to obtain the position markers of each element in the text to be evaluated and the target prediction evaluation results for at least two text evaluation dimensions; based on the position markers of each element in the text to be evaluated and the target prediction evaluation results for at least two text evaluation dimensions, determine the evaluation results of the text to be evaluated in at least two text evaluation dimensions.

[0075] In this context, each element in the text to be evaluated represents a sentence within that text.

[0076] Specifically, after the text to be evaluated is input into the fusion evaluation model, the position label of each sentence in the text to be evaluated will be predicted, for example, B i represents the beginning of the i-th sentence, E i represents the end of the i-th sentence, I i represents the middle of the i-th sentence; in addition, the fusion evaluation model also predicts the target predicted evaluation results of at least two text evaluation dimensions corresponding to each sentence in the text to be evaluated, for example, the i-th sentence is both action description and psychological description.

[0077] Based on this, after predicting the position label of each sentence in the text to be evaluated and the target predicted evaluation results of at least two text evaluation dimensions, the evaluation results of the entire text to be evaluated in at least two text evaluation dimensions can be further determined based on the position label of each sentence in the text to be evaluated and the target predicted evaluation results of at least two text evaluation dimensions corresponding to each sentence.

[0078] It should be understood that the number of sentences with at least two text evaluation dimensions in different texts to be evaluated is also different, for example, the number of sentences that are both action description and psychological description in the text to be evaluated A is 5, and the number of sentences that are both action description and psychological description in the text to be evaluated B is 3. Therefore, under the same other evaluation conditions, the score of the text to be evaluated A will be higher.

[0079] In this embodiment, by inputting the text to be evaluated into the fusion evaluation model, the position label of each element in the text to be evaluated and the target predicted evaluation results of at least two text evaluation dimensions are obtained, and then based on the position label of each element in the text to be evaluated and the target predicted evaluation results of at least two text evaluation dimensions, the number distribution of elements in at least two text evaluation dimensions in the text to be evaluated is obtained, and thus a more accurate evaluation result of at least two text evaluation dimensions of the text to be evaluated can be determined, thereby improving the accuracy of the text evaluation result of the text to be evaluated.

[0080] Further, it needs to be explained that in the prior art, for different text evaluation tasks, different loss functions need to be designed in the prior art to harmonize the weights of different text evaluation task loss functions in subsequent multi-task training. For example, assuming that the loss functions of the three tasks are l1, l2 and l3, the final loss function loss is λ1l1+λ2l2+λ3l3, and the specific values of the weights λ1, λ2 and λ3 are not currently implemented. The usual way is grid traversal, that is, all possibilities are traversed to find the best combination, and this way the training efficiency of the model is low.

[0081] To solve this problem, in the embodiment of the present application, when the initial fusion evaluation model is iteratively trained based on a plurality of first text samples without annotation information and at least two target evaluation results corresponding to each first text sample, the first text sample can be input into the initial fusion evaluation model to obtain the annotation sequence of the first text sample, the annotation sequence including the position mark of each element in the first text sample and the first prediction evaluation result of at least two text evaluation dimensions; the at least two target evaluation results corresponding to the first text sample are taken as the label information of the first text sample, and the initial fusion evaluation model is iteratively trained based on the label information of the first text sample and the corresponding annotation sequence to obtain the fusion evaluation model.

[0082] Specifically, Figure 4 The sequence annotation diagram of the first text sample provided by the embodiment of the present application is shown in the embodiment, in which the grammar correction task and the sentence classification task belonging to the sequence annotation task can be unified as a sequence annotation task. As shown in Figure 4 The grammar correction task is to predict whether each word is wrong, for example: the correction model identifies that “hapy” should be modified to “happy”, which is a spelling error. The detail description and highlight sentence identification task are both sentence classification tasks, and the usual practice is to take the representation of the BERT [CLS] position and then connect the fully connected network, and then classify. In the embodiment, the task form can be transformed, and the sequence annotation task of the entire sentence can be extracted. Specifically, after the first text sample is input into the initial fusion evaluation model, the initial fusion evaluation model will output the annotation sequence of the first text sample, which includes the position mark of each element in the first text sample and the first prediction evaluation result of at least two text evaluation dimensions. It should be understood that the position mark of each element can be understood as B, E or I, B represents the beginning of the sentence, E represents the end of the sentence, and I represents the middle of the sentence. The first prediction evaluation result includes whether there is a spelling or grammar error, whether it is a complex sentence or whether it is a detail description, etc. It should be understood that the first prediction evaluation result obtained through the above sequence annotation task is for each element in the first text sample.

[0083] For example, after the first text sample is input into the initial fusion evaluation model, it will be predicted that the position mark of the first element of the sentence is B, and the first prediction evaluation result is action description, the position mark of the last element of the sentence is E, and the first prediction evaluation result is action description, and the position mark of the middle element of the sentence is I, and the first prediction evaluation result is action description. That is, after the sequence annotation task of the entire sentence is extracted, if a sentence is an action description, the entire content of the sentence from the beginning to the end can be marked as an action description.

[0084] After obtaining the labeled sequence, at least two target evaluation results corresponding to the first text sample can be taken as label information of the first text sample, so as to determine loss information with the prediction result predicted by the initial fusion model, and the initial fusion evaluation model is iteratively trained based on the loss information, and the obtained initial fusion evaluation model is taken as the fusion evaluation model when the convergence condition is met.

[0085] In the embodiment, since the plurality of text evaluation tasks corresponding to the first text sample can be unified as a sequence labeling task, the determination of the weight hyperparameters of different loss functions is not required in the training process, and thus the training efficiency of the fusion evaluation model can be greatly improved.

[0086] Further, since the first prediction evaluation result obtained in the foregoing corresponds to each element in the first text sample, and in the actual implementation process, the prediction result for the first text sample is required when the initial fusion evaluation model is trained, therefore, when the initial fusion evaluation model is iteratively trained, the second prediction evaluation result of the first text sample can be determined based on the position mark of each element in the first text sample and the first prediction evaluation result of at least two text evaluation dimensions, and the loss information of the first text sample is determined based on the label information and the second prediction evaluation result of the first text sample, so that the initial fusion evaluation model is iteratively trained based on the loss information of the first text sample, and the fusion evaluation model is obtained.

[0087] Since the position mark of each element in the first text sample and the first prediction evaluation result of each text evaluation dimension are labeled when sequence labeling is performed, the complete sentence can be determined based on the position mark of each element, and the prediction evaluation result of the sentence can be determined based on the first prediction evaluation result of each element in the sentence, so that the second prediction result of the first text sample can be determined. For example, after sequence labeling, if the first prediction evaluation result corresponding to the first element of the first text sample is action description, the first prediction evaluation result corresponding to the last element is action description, and the first prediction evaluation result corresponding to the middle element is action description, it can be determined that the probability of the second prediction evaluation result of the first text sample being action description is high.

[0088] After the plurality of text evaluation tasks corresponding to the first text sample are unified as a sequence labeling task, a unified loss function shown in the following formula (1) can be used. For example, a Sigmoid cross entropy loss function can be used to calculate the loss information Loss of the first text sample.

[0089] Loss = -y*log(sigmoid(x))-(1-y)*log(1-sigmoid(x)) (1)

[0090] Wherein, y represents the label information of the first text sample, sigmoid(x) represents the second prediction result of the first text sample, that is, the probability of the first text sample belonging to the label.

[0091] On this basis, the loss information Loss of the first text sample calculated by the above formula (1) can be used to iteratively train the initial fusion evaluation model, that is, the weight of the parameters in the initial fusion evaluation model is updated using the back propagation algorithm, so as to reduce the loss information Loss of the first text sample, until the loss information is less than a certain preset loss threshold, or the model converges, thereby obtaining the final fusion evaluation model.

[0092] In the embodiment, by unifying the plurality of text evaluation tasks corresponding to the first text sample into a sequence labeling task, the loss function form of the plurality of text evaluation tasks can be unified, thereby avoiding the weight adjustment of the loss functions of different text evaluation tasks, that is, without the need to design the weight hyperparameters of different loss functions. Therefore, unnecessary calculation processes and variable introduction are reduced, thereby not only improving the accuracy of the fusion evaluation model, but also improving the training efficiency of the fusion evaluation model.

[0093] Figure 5 The flowchart of the evaluation method of the text content provided in the embodiment is shown in Figure 2. The embodiment details the specific implementation process of how to obtain at least two target evaluation results corresponding to the first text sample when at least two target content evaluation models are included in each type of content evaluation model based on the above embodiments. As shown in the figure, the method comprises: Figure 5

[0094] Step 501: obtaining a plurality of first text samples without annotation information.

[0095] Wherein, the implementation process of step 501 is similar to that of step 201, and the specific implementation process is described in detail in step 201, which will not be repeated here.

[0096] Step 502: inputting the first text sample into at least two target content evaluation models corresponding to the content evaluation model for each type of content evaluation model, to obtain at least two initial evaluation results corresponding to the first text sample.

[0097] ​It should be understood that predictions obtained using multiple target content evaluation models are generally better than those obtained using a single target content evaluation model. Therefore, to improve the accuracy of target evaluation results, various content evaluation models may include at least two target content evaluation models.

[0098] Among the various content evaluation models, the at least two target content evaluation models can have the same or different model structures. In practical applications, regardless of whether the structures of the at least two target content evaluation models are the same, they are usually set to a unified sequence labeling model. Given that the structures of the at least two target content evaluation models are determined, different models can be obtained with the same training data by replacing the random seed. Under different random seeds, the initial values ​​of the model parameters are different, thus resulting in different ultimately trained content evaluation models. Alternatively, different pre-trained models can be used. Besides BERT, RoBERTa, XLNET, and DeBERTa (Decoding-enhanced BERT with disentangled Attention) model structures can also be selected.

[0099] For example, Figure 6 This is a second schematic diagram illustrating the generation of target evaluation results provided in an embodiment of the present invention, as shown below. Figure 6 As shown, there are three types of content evaluation models. Task 1 content evaluation models include Task 1 target content evaluation model 1, Task 1 target content evaluation model 2, and Task 1 target content evaluation model 3. Task 2 and Task 3 content evaluation models are similar. After inputting the first unlabeled text sample into the three target content evaluation models corresponding to each type of content evaluation model, the initial evaluation result output by each target content evaluation model can be obtained. For example, Task 1 target content evaluation model 1 outputs Task 1 initial evaluation result 1, Task 1 target content evaluation model 2 outputs Task 1 initial evaluation result 2, and Task 1 target content evaluation model 3 outputs Task 1 initial evaluation result 3, and so on.

[0100] Step 503: Based on at least two initial evaluation results corresponding to the first text sample, determine the target evaluation result corresponding to the first text sample.

[0101] Specifically, after obtaining at least two initial evaluation results for the first text sample based on various content evaluation models, the at least two initial evaluation results for each type of content evaluation model can be filtered according to preset filtering rules to obtain the target evaluation result for the first text sample. In the specific implementation process, the target evaluation result can be determined through voting or weighting methods.

[0102] In a possible implementation, when the number of target content evaluation models is at least three, and the target evaluation result corresponding to the first text sample is determined based on at least two initial evaluation results corresponding to the first text sample, the initial evaluation result with the highest proportion in the at least three initial evaluation results corresponding to the first text sample can be determined as the target evaluation result corresponding to the first text sample.

[0103] For example, Figure 6 Taking task 1 as an example, for a plurality of first text samples without labeled information, the first text sample is input into the three target content evaluation models corresponding to task 1, and three initial evaluation results corresponding to the first text sample can be obtained. The initial evaluation result with the highest proportion in the three initial evaluation results (two identical evaluation results) is taken as the target evaluation result corresponding to the first text sample, that is, the idea of minority obeying majority (voting) is adopted, and the plurality of initial evaluation results of the first text sample are fused.

[0104] Specifically, if at least n-1 initial evaluation results in n initial evaluation results are consistent, the same initial evaluation result predicted by the n-1 models is retained and taken as the target evaluation result of the first text sample. For example, it is assumed that Figure 6 In the example, the initial evaluation results of the three task 1 target content evaluation models on the first text sample “I am hapy that you come.” are hapy->happy, hapy->happy, and hapy->good. It can be seen that two initial evaluation results in the three initial evaluation results are consistent, and at this time, it is considered that the initial evaluation result of the first text sample is accurate, that is, hapy->happy is taken as the target evaluation result, and other initial evaluation results are discarded or excluded.

[0105] Similarly, continuing to refer to Figure 6In task 2 and task 3, the first text sample is also evaluated by the three target content evaluation models, and three initial evaluation results are obtained respectively, and the target evaluation results of task 2 and task 3 are determined by voting. In this way, the target evaluation results of 3 tasks can be obtained on any first text sample without labeled information, so as to be applied to the training of the subsequent fusion evaluation model. For example, assuming that the first text sample is "I am hapy that you come", the target evaluation result corresponding to task 1 is "hapy->happy"; the target evaluation result of task 2 is "object clause"; and the target evaluation result of task 3 is "psychological description". Based on this, the three target evaluation results obtained can be combined to obtain the label information of the first text sample as "hapy-happy", "object clause" and "psychological description".

[0106] The difference between the above-mentioned manner and the labeled training data is that each text sample in the labeled training data usually has only one task annotation information, and each first text sample evaluated by the above-mentioned multiple target content evaluation models has annotation information of all tasks, that is, the label information of the first text sample.

[0107] In the embodiment, when the number of target evaluation models in each type of content evaluation model is at least three, the first text sample can obtain at least three initial evaluation results when facing different text evaluation tasks, and the initial evaluation result with the highest proportion is taken as the target evaluation result corresponding to the first text sample. In this way, the target evaluation result as the label information of the first text sample can be more accurate, and the accuracy of the fusion evaluation model can be improved, thereby improving the accuracy of text evaluation.

[0108] Step 504: based on the multiple first text samples without labeled information and the at least two target evaluation results corresponding to each first text sample, iteratively training the initial fusion evaluation model to obtain a fusion evaluation model, and the fusion evaluation model is used to evaluate the text to be evaluated from at least two text evaluation dimensions.

[0109] Among them, the implementation process of step 504 is similar to that of step 202, and the specific implementation process can refer to the related description in step 202, which will not be repeated here.

[0110] In the training method of the text content evaluation model provided in this embodiment, when the number of target content evaluation models among various content evaluation models is at least two, the first text sample can obtain at least two initial evaluation results when facing each text evaluation task. This allows for the selection of a more accurate initial evaluation result from the at least two initial evaluation results, which is then used as the target evaluation result for the first text sample. This makes the target evaluation result for the label information of the first text sample more accurate, improving the accuracy of the fusion evaluation model.

[0111] Furthermore, the content evaluation model mentioned in the above embodiments can be trained in the following way: obtain multiple second text samples with labeled information, wherein the labeled information of each second text sample is obtained after evaluating the second text sample based on the target text evaluation dimension, and the target text evaluation dimension is any one of at least two text evaluation dimensions; based on each second text sample and the corresponding labeled information, iteratively train the initial content evaluation model to obtain a content evaluation model corresponding to the target text evaluation dimension.

[0112] For example, Figure 7 This is a training diagram of the content evaluation model provided in an embodiment of the present invention, such as... Figure 7 As shown, after evaluating the second text samples through different target text evaluation dimensions, second text samples with different annotation information can be obtained. By classifying these second text samples with different annotation information, the second text samples corresponding to the annotation information of the same text evaluation task can be obtained, i.e. Figure 7 The diagram shows the second text samples corresponding to Task 1, Task 2, and Task 3. Based on these second text samples and their corresponding annotations for each text evaluation task, the initial content evaluation model is iteratively trained. This involves updating the parameter weights in the model using the BackPropagation (BP) algorithm, thereby obtaining the content evaluation model corresponding to that text evaluation task. For example, the initial content evaluation model for Task 1 can be iteratively trained using the second text samples corresponding to Task 1 to obtain the content evaluation model for Task 1. The training process for the content evaluation models for Tasks 2 and 3 is similar to that for Task 1 and will not be elaborated further here.

[0113] It should be noted that the different tasks mentioned above can correspond to different text evaluation dimensions. For example, Task 1 corresponds to text evaluation dimension 1, such as whether there are spelling errors; Task 2 corresponds to text evaluation dimension 2, such as whether there are complex sentence structures; and Task 3 corresponds to text evaluation dimension 3, such as whether there are sentences with detailed descriptions, etc.

[0114] In the embodiment, the initial content evaluation model is iteratively trained by the second text sample with the annotation information and the corresponding annotation information, and a more accurate content evaluation model corresponding to the target text evaluation dimension is obtained. In this way, when the content evaluation model is used to determine the target evaluation result of the first text sample in the subsequent, the accuracy of the target evaluation result can be improved, thereby providing a basis for obtaining a more accurate fusion evaluation model in the subsequent.

[0115] Further, on the basis of the above embodiment, in order to improve the accuracy of the subsequent target evaluation result, at least three models can be included in the initial content evaluation model, and at least three target content evaluation models corresponding to the target text evaluation dimension can be obtained when the initial content evaluation model is iteratively trained based on the second text sample and the corresponding annotation information.

[0116] Among them, the target content evaluation model in each type of initial content evaluation model can be the same model structure or different model structure, and no specific limitation is made to this.

[0117] Exemplarily, Figure 8 The training schematic diagram of the target content evaluation model provided by the embodiment of the present application is shown in FIG. 8. After the second text sample is annotated and classified, the second text sample corresponding to different text evaluation tasks is obtained, and then the three initial target content evaluation models of the same text evaluation task can be iteratively trained based on the second text sample corresponding to the same text evaluation task, so as to obtain three target content evaluation models of each type of text evaluation task. The iterative training method is the same as the above embodiment, and will not be repeated here.

[0118] In the embodiment, by setting the number of initial target content evaluation models in each type of initial content evaluation model to at least three, when the second text sample faces the same text evaluation task, at least three initial evaluation results can be output based on at least three target content evaluation models, so that the target evaluation result can be determined based on at least three initial evaluation results. In this way, the accuracy of the target evaluation result can be improved, which provides a basis for the training of the fusion evaluation model and realizes higher-accuracy text evaluation.

[0119] The text content evaluation device provided by the embodiment of the present application is described below. The text content evaluation device described below can be referred to each other corresponding to the text content evaluation method described above.

[0120] Figure 9 The structure schematic diagram of the text content evaluation device provided by the embodiment of the present application is shown in FIG. Figure 9 The device comprises:

[0121] The acquisition module 901 is configured to acquire a text to be evaluated.

[0122] The input module 902 is configured to input the text to be evaluated into a fusion evaluation model to obtain evaluation results of at least two text evaluation dimensions, wherein the fusion evaluation model is obtained by iteratively training an initial fusion evaluation model based on a plurality of first text samples without annotation information and at least two target evaluation results corresponding to each of the first text samples, and the at least two target evaluation results are obtained by inputting each of the first text samples into at least two content evaluation models.

[0123] The fusion evaluation model in the text content evaluation device provided by the embodiment of the present application is obtained by iteratively training an initial fusion evaluation model based on a plurality of first text samples without annotation information acquired by the acquisition module 901 and at least two target evaluation results corresponding to each of the first text samples, and the at least two target evaluation results are obtained by inputting each of the first text samples into at least two content evaluation models through the input module 902. Therefore, in the process of iteratively training the initial fusion evaluation model based on the first text samples and the at least two target evaluation results corresponding to each of the first text samples, the target evaluation results of at least two text evaluation dimensions corresponding to the same first text sample can be used as labels to iteratively train the initial fusion evaluation model, so that the difference between the number of first text samples corresponding to different text evaluation dimensions is small, thereby avoiding the problem that the fusion evaluation model is inaccurate due to a large difference in the amount of training data samples of text evaluation tasks corresponding to different text evaluation dimensions when the fusion evaluation model is trained, improving the accuracy of the fusion evaluation model, and further inputting the acquired text to be evaluated into the more accurate fusion evaluation model to obtain more accurate text evaluation results.

[0124] Optionally, the input module 902 is specifically configured to:

[0125] input the text to be evaluated into the fusion evaluation model to obtain position markers of each element in the text to be evaluated and target predicted evaluation results of at least two text evaluation dimensions;

[0126] determine the evaluation results of the text to be evaluated in the at least two text evaluation dimensions based on the position markers of each element in the text to be evaluated and the target predicted evaluation results of the at least two text evaluation dimensions.

[0127] Optionally, the input module 902 is further configured to input the first text sample into the initial fusion evaluation model to obtain a labeled sequence of the first text sample, and the labeled sequence includes position markers of each element in the first text sample and first predicted evaluation results of at least two text evaluation dimensions;

[0128] The training module is configured to take the at least two target evaluation results corresponding to the first text sample as label information of the first text sample, and perform iterative training on the initial fusion evaluation model based on the label information of the first text sample and the corresponding annotation sequence to obtain the fusion evaluation model.

[0129] Optionally, the training module is specifically configured to:

[0130] determine a second predicted evaluation result of the first text sample based on the position mark of each element in the first text sample and the first predicted evaluation result of the at least two text evaluation dimensions;

[0131] determine loss information of the first text sample based on the label information of the first text sample and the second predicted evaluation result;

[0132] perform iterative training on the initial fusion evaluation model based on the loss information of the first text sample to obtain the fusion evaluation model.

[0133] Optionally, the various content evaluation models include at least two target content evaluation models; the input module 902 is specifically configured to

[0134] input the first text sample into the at least two target content evaluation models corresponding to the content evaluation model to obtain at least two initial evaluation results corresponding to the first text sample;

[0135] determine the target evaluation result corresponding to the first text sample based on the at least two initial evaluation results corresponding to the first text sample.

[0136] Optionally, the number of target content evaluation models is at least three.

[0137] The input module 902 includes:

[0138] The determination unit is configured to determine, from the at least three initial evaluation results corresponding to the first text sample, an initial evaluation result with the highest proportion as the target evaluation result corresponding to the first text sample.

[0139] Optionally, the acquisition module 901 is further configured to acquire a plurality of second text samples with annotation information, and the annotation information of each second text sample is obtained based on evaluation of the second text sample by a target text evaluation dimension, the target text evaluation dimension being any one of the at least two text evaluation dimensions.

[0140] The training module is further configured to perform iterative training on the initial content evaluation model based on each second text sample and the corresponding annotation information to obtain a content evaluation model corresponding to the target text evaluation dimension.

[0141] The apparatus of this embodiment can be used to execute the method of any embodiment of the text content evaluation device side method embodiment. Its specific implementation process and technical effects are similar to those of the text content evaluation device side method embodiment. For details, please refer to the detailed description in the text content evaluation device side method embodiment, which will not be repeated here.

[0142] Figure 10 This example illustrates one of the physical structural diagrams of an electronic device, such as... Figure 10 As shown, the electronic device may include a processor 1001, a communications interface 1002, a memory 1003, and a communication bus 1004. The processor 1001, communications interface 1002, and memory 1003 communicate with each other via the communication bus 1004. The processor 1001 can call logical instructions in the memory 1003 to execute a text content evaluation method. This method includes: acquiring the text to be evaluated; inputting the text to be evaluated into a fusion evaluation model to obtain evaluation results for at least two text evaluation dimensions. The fusion evaluation model is obtained by iteratively training an initial fusion evaluation model based on multiple unlabeled first text samples and at least two target evaluation results corresponding to each first text sample. The at least two target evaluation results are obtained by inputting each first text sample into at least two types of content evaluation models.

[0143] Furthermore, the logical instructions in the aforementioned memory 1003 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0144] Figure 11 Example 2: A schematic diagram of the physical structure of an electronic device, such as... Figure 11As shown, the electronic device can include a processor 1101, a communications interface 1102, a memory 1103, and a communications bus 1104, and further include a display screen 1105, wherein the processor 1101, the communications interface 1102, the memory 1103, and the display screen 1105 complete mutual communication through the communications bus 1104. The display screen 1105 is used to display the evaluation result, and the processor 1101 can call the logical instructions in the memory 1103 to execute the fusion evaluation model in the computer program to process the text to be evaluated to obtain the evaluation result of at least two text evaluation dimensions; wherein the fusion evaluation model is obtained by iteratively training an initial fusion evaluation model based on a plurality of first text samples without annotation information and at least two target evaluation results corresponding to each of the first text samples, and the at least two target evaluation results are obtained by inputting each of the first text samples into at least two content evaluation models.

[0145] In addition, the logical instructions in the memory 1103 described above can be implemented in the form of a software functional unit and sold or used as an independent product, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, includes a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0146] On the other hand, the present application also provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer readable storage medium, when the computer program is executed by a processor, the computer can execute the text content evaluation method provided by the above-mentioned method, the method includes: obtaining the text to be evaluated; inputting the text to be evaluated into the fusion evaluation model to obtain the evaluation result of at least two text evaluation dimensions, wherein the fusion evaluation model is obtained by iteratively training an initial fusion evaluation model based on a plurality of first text samples without annotation information and at least two target evaluation results corresponding to each of the first text samples, and the at least two target evaluation results are obtained by inputting each of the first text samples into at least two content evaluation models.

[0147] In yet another aspect, the present application also provides a non-transitory computer readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the method for evaluating text content as provided by any of the above methods, the method comprising: obtaining a text to be evaluated; inputting the text to be evaluated into a fusion evaluation model to obtain evaluation results of at least two text evaluation dimensions, wherein the fusion evaluation model is obtained by iteratively training an initial fusion evaluation model based on a plurality of first text samples without labeled information and at least two target evaluation results corresponding to each of the first text samples, and the at least two target evaluation results are obtained by inputting each of the first text samples into at least two content evaluation models.

[0148] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place or distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0149] From the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be realized by means of software plus a necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0150] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of evaluating text content, characterized by, The method comprises: obtaining a text to be evaluated; inputting the text to be evaluated into a fusion evaluation model to obtain evaluation results of at least two text evaluation dimensions, wherein the fusion evaluation model is obtained by iteratively training an initial fusion evaluation model based on a plurality of first text samples without annotation information and at least two target evaluation results corresponding to each of the first text samples, and the at least two target evaluation results are obtained by inputting each of the first text samples into at least two content evaluation models; the fusion evaluation model is used to evaluate the text to be evaluated from at least two text evaluation dimensions, different text evaluation dimensions correspond to different evaluation criteria, and the text evaluation dimensions include at least two of spelling errors, complex sentence patterns, and detailed descriptions; the inputting of the text to be evaluated into the fusion evaluation model to obtain the evaluation results of the at least two text evaluation dimensions comprises: inputting the text to be evaluated into the fusion evaluation model to obtain position markers of each element in the text to be evaluated and target prediction evaluation results of at least two text evaluation dimensions corresponding to each element; the position markers are used to indicate positions of each element with at least two text evaluation dimensions in the text to be evaluated, each element represents each sentence in the text to be evaluated, and the position markers include a beginning position, an end position, or an intermediate position of a sentence; determining the evaluation results of the text to be evaluated in at least two text evaluation dimensions based on the position markers of each element in the text to be evaluated and the target prediction evaluation results of at least two text evaluation dimensions corresponding to each element; the iteratively training of the initial fusion evaluation model based on the plurality of first text samples without annotation information and the at least two target evaluation results corresponding to each of the first text samples to obtain the fusion evaluation model comprises: inputting the first text samples into the initial fusion evaluation model to obtain a label sequence of the first text samples, wherein the label sequence includes position markers of each element in the first text samples and first prediction evaluation results of at least two text evaluation dimensions; using the at least two target evaluation results corresponding to the first text samples as label information of the first text samples, and iteratively training the initial fusion evaluation model based on the label information of the first text samples and the corresponding label sequence to obtain the fusion evaluation model.

2. The evaluation method of text content according to claim 1, characterized in that, the iteratively training of the initial fusion evaluation model based on the label information of the first text samples and the corresponding label sequence to obtain the fusion evaluation model comprises: determining second prediction evaluation results of the first text samples based on the position markers of each element in the first text samples and the first prediction evaluation results of at least two text evaluation dimensions; determining loss information of the first text samples based on the label information and the second prediction evaluation results of the first text samples; iteratively training the initial fusion evaluation model based on the loss information of the first text samples to obtain the fusion evaluation model.

3. The method of evaluating the content of text according to claim 1 or 2, characterized in that, The content evaluation models include at least two target content evaluation models; The at least two target evaluation results corresponding to the first text sample are obtained based on the following manner: For each type of content evaluation model, the first text sample is input into at least two target content evaluation models corresponding to the content evaluation model to obtain at least two initial evaluation results corresponding to the first text sample; Based on the at least two initial evaluation results corresponding to the first text sample, the target evaluation result corresponding to the first text sample is determined.

4. The method of evaluating text content according to claim 3, wherein, The number of target content evaluation models is at least three; The target evaluation result corresponding to the first text sample is determined based on the at least two initial evaluation results corresponding to the first text sample, including: The initial evaluation result with the highest proportion in the at least three initial evaluation results corresponding to the first text sample is determined as the target evaluation result corresponding to the first text sample.

5. The method of evaluating text content according to claim 1 or 2, characterized in that, The method further includes: Obtaining a plurality of second text samples with labeled information, the labeled information of each second text sample being obtained by evaluating the second text sample based on a target text evaluation dimension, and the target text evaluation dimension being any one of the at least two text evaluation dimensions; Based on each second text sample and the corresponding labeled information, an initial content evaluation model is iteratively trained to obtain a content evaluation model corresponding to the target text evaluation dimension.

6. An evaluation device of text content, characterized by, It includes: An acquisition module is configured to acquire a text to be evaluated; An input module is configured to input the text to be evaluated into a fusion evaluation model to obtain evaluation results of at least two text evaluation dimensions, wherein the fusion evaluation model is obtained by iteratively training an initial fusion evaluation model based on a plurality of first text samples without labeled information and at least two target evaluation results corresponding to each first text sample, and the at least two target evaluation results are obtained by inputting each first text sample into at least two types of content evaluation models; The fusion evaluation model is used to evaluate the text to be evaluated from at least two text evaluation dimensions, different text evaluation dimensions correspond to different evaluation criteria, and the text evaluation dimensions include at least two of whether there is a spelling error, whether there is a complex sentence, and whether there is a detailed description; The input module is configured to input the text to be evaluated into the fusion evaluation model to obtain position markers of each element in the text to be evaluated and target predicted evaluation results of at least two text evaluation dimensions corresponding to each element, wherein the position markers are used to indicate positions of each element with at least two text evaluation dimensions in the text to be evaluated, each element represents each sentence in the text to be evaluated, and the position markers include a beginning position, an end position, or an intermediate position of a sentence. ​ determine the evaluation result of the to-be-evaluated text in at least two text evaluation dimensions based on the position mark of each element in the to-be-evaluated text and the target predicted evaluation result of each element in at least two text evaluation dimensions; The initial fusion evaluation model is iteratively trained based on the plurality of first text samples without annotation information and the at least two target evaluation results corresponding to each first text sample, to obtain the fusion evaluation model, including: The first text sample is input into the initial fusion evaluation model to obtain a label sequence of the first text sample, and the label sequence includes the position mark of each element in the first text sample and the first predicted evaluation result of at least two text evaluation dimensions; The at least two target evaluation results corresponding to the first text sample are taken as label information of the first text sample, and the initial fusion evaluation model is iteratively trained based on the label information of the first text sample and the corresponding label sequence, to obtain the fusion evaluation model.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the text content evaluation method of any one of claims 1 to 5.

8. An electronic device comprising a display screen, further comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the fusion evaluation model in the computer program to process the to-be-evaluated text to obtain the evaluation result of at least two text evaluation dimensions; The display screen is configured to display the evaluation result. The fusion evaluation model is obtained by iteratively training an initial fusion evaluation model based on a plurality of first text samples without annotation information and at least two target evaluation results corresponding to each first text sample, and the at least two target evaluation results are obtained by inputting each first text sample into at least two content evaluation models; The fusion evaluation model is configured to evaluate the to-be-evaluated text from at least two text evaluation dimensions, different text evaluation dimensions correspond to different evaluation criteria, and the text evaluation dimensions include at least two of spelling errors, complex sentence patterns, and detailed descriptions; The fusion evaluation model is configured to evaluate the to-be-evaluated text from at least two text evaluation dimensions, different text evaluation dimensions correspond to different evaluation criteria, and the text evaluation dimensions include at least two of spelling errors, complex sentence patterns, and detailed descriptions; The fusion evaluation model is configured to evaluate the to-be-evaluated text from at least two text evaluation dimensions, different text evaluation dimensions correspond to different evaluation criteria, and the text evaluation dimensions include at least two of spelling errors, complex sentence patterns, and detailed descriptions; The to-be-evaluated text is input into the fusion evaluation model to obtain the position mark of each element in the to-be-evaluated text and the target predicted evaluation result of each element in at least two text evaluation dimensions, and the position mark is used to indicate the position of each element with at least two text evaluation dimensions in the to-be-evaluated text, each element represents each sentence in the to-be-evaluated text, and the position mark includes a beginning position, an end position, or an intermediate position in a sentence; determine the evaluation result of the to-be-evaluated text in at least two text evaluation dimensions based on the position mark of each element in the to-be-evaluated text and the target predicted evaluation result of each element in at least two text evaluation dimensions; The initial fusion evaluation model is iteratively trained based on the plurality of first text samples without annotation information and the at least two target evaluation results corresponding to each first text sample, to obtain the fusion evaluation model, including: inputting the first text sample into the initial fusion evaluation model to obtain a label sequence of the first text sample, the label sequence including position marks of each element in the first text sample and first predicted evaluation results of at least two text evaluation dimensions; taking at least two target evaluation results corresponding to the first text sample as label information of the first text sample, and performing iterative training on the initial fusion evaluation model based on the label information of the first text sample and the corresponding label sequence to obtain the fusion evaluation model. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by a processor to implement the text content evaluation method according to any one of claims 1 to 5.

Images