Text evaluation method, device, computer-readable storage medium, and terminal device

Abstract

The application belongs to the technical field of text processing, and particularly relates to a text evaluation method and device, a computer readable storage medium and a terminal device. The method comprises the following steps: obtaining a target text to be evaluated; and processing the target text by using a trained text evaluation model to obtain a score result of the target text; wherein the text evaluation model is a model obtained by fine-tuning a mask language model based on text data labeled from a plurality of preset evaluation dimensions. Through the application, the text data labeled from multiple dimensions is further fine-tuned on the trained mask language model, the composition is evaluated from multiple dimensions, effective assistance is provided for teachers to correct compositions and for students to practice writing, the correction efficiency of teachers can be improved, and the writing practice effect of students can be improved.

Description

Technical Field

[0001] The present application belongs to the field of text processing technology, and in particular relates to a text evaluation method, apparatus, computer-readable storage medium, and terminal device. Background Art

[0002] Correcting large numbers of student essays often consumes a significant amount of teachers' time. To reduce this burden, it's essential to provide effective essay assessment tools. Students often lack effective essay assessment tools and targeted improvement suggestions during their writing practice, resulting in unsatisfactory results. Summary of the Invention

[0003] In view of this, the embodiments of the present application provide a text evaluation method, apparatus, computer-readable storage medium, and terminal device to solve the problems in the prior art of teachers being overburdened with grading essays and students lacking effective means of essay evaluation.

[0004] A first aspect of the embodiments of the present application provides a text evaluation method, which may include:

[0005] Get the target text to be evaluated;

[0006] The target text is processed using a trained text evaluation model to obtain a scoring result for the target text; wherein the text evaluation model is a model obtained by fine-tuning a masked language model based on text data annotated from several preset evaluation dimensions.

[0007] In a possible implementation of the first aspect, processing the target text using the trained text evaluation model to obtain a scoring result for the target text includes:

[0008] Calculating the probability distribution of the target text at different scoring levels of the plurality of evaluation dimensions;

[0009] For each evaluation dimension, the rating level with the maximum probability in the probability distribution is used as the rating result of the target text in the evaluation dimension.

[0010] In a possible implementation of the first aspect, calculating the probability distribution of the target text at different scoring levels of the plurality of evaluation dimensions includes:

[0011] Obtaining encoding information of the target text;

[0012] Converting the encoded information according to a preset algorithm to obtain a conversion result of the target text;

[0013] According to the conversion processing result, the probability distribution of the target text at different scoring levels of the plurality of evaluation dimensions is calculated.

[0014] In a possible implementation of the first aspect, calculating, based on the conversion processing result, the probability distribution of the target text at different scoring levels of the plurality of evaluation dimensions, includes:

[0015] Taking the mean of the first dimension of the conversion processing result to obtain a mean vector of the conversion processing result;

[0016] Performing a linear transformation on the mean vector to obtain a linear transformation result of the mean vector;

[0017] The linear transformation results are subjected to softmax transformation in different evaluation dimensions to obtain the probability distribution of the target text at different scoring levels in the several evaluation dimensions.

[0018] In a possible implementation of the first aspect, obtaining the encoding information of the target text includes:

[0019] Obtain a word embedding matrix and an index of each word in the target text in a dictionary, wherein the word embedding matrix is ​​a matrix corresponding to the words in the dictionary;

[0020] Determining an embedding code of the target text according to the word embedding matrix and the index of each word in the target text in the dictionary;

[0021] Obtaining a position code of the target text, wherein the position code is determined according to relative position information of each word in the target text;

[0022] The encoding information of the target text is determined according to the embedded code and the position code.

[0023] In a possible implementation of the first aspect, converting the encoded information according to a preset algorithm to obtain a conversion result of the target text includes:

[0024] Performing conversion processing on the encoded information based on the attention mechanism to obtain an initial conversion result;

[0025] Performing residual connection processing on the encoded information and the initial conversion processing result to obtain a residual connection result;

[0026] The residual connection result is fully connected through a fully connected layer to obtain a conversion processing result of the target text.

[0027] In a possible implementation of the first aspect, the multiple evaluation dimensions include: grammatical rationality, richness of vocabulary, spelling correctness, and logical coherence.

[0028] A second aspect of the embodiments of the present application provides a text evaluation device, which may include:

[0029] A text acquisition unit, used to acquire a target text to be evaluated;

[0030] A text evaluation unit is used to process the target text using a trained text evaluation model to obtain a scoring result for the target text; wherein the text evaluation model is a model obtained by fine-tuning a masked language model based on text data annotated from several preset evaluation dimensions.

[0031] A third aspect of the embodiments of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned text evaluation methods are implemented.

[0032] A fourth aspect of an embodiment of the present application provides a terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any one of the above-mentioned text evaluation methods when executing the computer program.

[0033] A fifth aspect of the embodiments of the present application provides a computer program product, which, when executed on a terminal device, enables the terminal device to execute the steps of any one of the above-mentioned text evaluation methods.

[0034] Compared with the prior art, the embodiments of the present application have the following advantages: The embodiments of the present application adopt a masking approach, supplemented by technical means such as parameter decomposition and parameter sharing. The language model trained on a large amount of text data can better learn the semantic representation of each word in the text with contextual information. Further fine-tuning of the trained masked language model on the text data annotated from multiple dimensions is performed, and essays are evaluated from multiple dimensions. This provides effective assistance for teachers to correct essays and students to practice writing, which can improve teachers' correction efficiency and improve students' writing practice results. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0036] Figure 1 This is a flow chart of an embodiment of a text evaluation method in an embodiment of the present application;

[0037] Figure 2 A schematic flowchart of the process of the text evaluation model processing the target text;

[0038] Figure 3 A schematic flow chart for calculating the probability distribution of target text at different rating levels of several evaluation dimensions;

[0039] Figure 4 This is a structural diagram of an embodiment of a text evaluation device in an embodiment of the present application;

[0040] Figure 5 This is a schematic block diagram of a terminal device in an embodiment of the present application. DETAILED DESCRIPTION

[0041] In order to make the purpose, features, and advantages of the invention of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the embodiments described below are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0042] It will be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0043] It should also be understood that the terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0044] It should be further understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0045] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0046] In addition, in the description of the present application, the terms "first", "second", "third", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0047] See also Figure 1 In the embodiments of the present application, an embodiment of a text evaluation method may include:

[0048] Step S101: Obtain the target text to be evaluated.

[0049] The target text can be specified by the user according to actual circumstances. For example, when a teacher is grading a composition, the composition to be graded can be specified as the target text; when a student is doing writing practice, his or her own writing can be specified as the target text.

[0050] Step S102: Use the trained text evaluation model to process the target text to obtain a scoring result of the target text.

[0051] The text evaluation model is a model obtained by fine-tuning a masked language model based on text data annotated from several preset evaluation dimensions.

[0052] The text evaluation model mainly consists of three parts, including an input layer, a conversion processing layer, and an output layer. Specifically, the text evaluation model is used to output a corresponding scoring result based on the input target text.

[0053] As a possible implementation of this application, Figure 2 The specific implementation process of the text evaluation model processing the target text in the text evaluation method provided in the embodiment of the present application is shown, and the following steps S201 to S202 are detailed:

[0054] Step S201: Calculate the probability distribution of the target text at different scoring levels of the plurality of evaluation dimensions.

[0055] As a possible implementation of this application, Figure 3 The specific implementation process of step S201 of the text evaluation method provided in the embodiment of the present application is shown and is detailed as follows:

[0056] A1: Obtain encoding information of the target text.

[0057] The above encoding information is obtained by encoding the sentences in the target text through the input layer of the text evaluation model. In the embodiment of the present application, the sentences are encoded by word embedding encoding and position encoding to obtain the encoding information of the sentences in the above target text.

[0058] In the embodiment of the present application, the encoding information of the sentence includes embedded coding and position coding.

[0059] As a possible implementation of the present application, the above step A1 specifically includes:

[0060] A11: Obtain a word embedding matrix and the index of each word in the target text in the dictionary.

[0061] The word embedding matrix is ​​a matrix corresponding to the words in the dictionary. In the embodiment of the present application, word segmentation is performed based on the text information in the training corpus to obtain the dictionary Vocabulary, where Vocabulary = {w1, w2, ..., w v},w1,w2,…,w v They are the words in the dictionary, v is the number of words in the dictionary Vocabulary, and a word embedding matrix M is randomly initialized using Gaussian distribution. v , m is the dimension of word vector. Word embedding matrix M v A row in corresponds to the initial word vector of the word in the corresponding position in the dictionary Vocabulary, and the word embedding matrix M v Continuously updated as the text evaluation model is continuously trained.

[0062] For a sentence S(w1,w2,…,w n ), n is the number of words in sentence S. By searching the vocabulary, we can get the index of each word in sentence S in the vocabulary.

[0063] A12: Determine an embedding code for the target text based on the word embedding matrix and the index of each word in the target text in the dictionary.

[0064] Specifically, based on the index of each word in the dictionary and the word embedding matrix in the above sentence, the word vector of each word in the sentence S can be obtained, and the word vectors of all words in the sentence S are concatenated to form a two-dimensional matrix, which represents the embedding code S of the sentence. emb ,

[0065] A13: Obtain the position code of the target text.

[0066] The position code is determined based on the relative position information of each word in the target text. In the embodiment of the present application, the position information of each word is not included in the embedded code. By introducing the position code, the position of the word in the sentence can be effectively identified. Specifically, the position code S is randomly initialized using a Gaussian distribution. pos , Position code S pos Continuously updated as the text evaluation model is continuously trained.

[0067] A14: Determine the encoding information of the target text based on the embedded code and the position code.

[0068] In the embodiment of the present application, the encoding information of the sentence is encoded by the embedded code S emb and position code S pos Determine. When training a text evaluation model, the general practice is to maintain the dimension m of the word vector equal to the size of the subsequent hidden layer (Hidden State). However, the word vector learns a context-independent semantic representation of the word, while the hidden layer often learns a context-dependent semantic representation of the word. The context of the word is crucial for text evaluation. To make the hidden layer more expressive and learn richer context, the hidden layer is often larger. Because the size of the word vector in the word embedding matrix is ​​equal to the size of the hidden layer, the number of parameters in the word embedding matrix is ​​very large, resulting in a huge number of parameters for the entire model.

[0069] In the embodiment of the present application, since the word embedding matrix M v It is a matrix with a large number of parameters. The value of m is generally large, by introducing an intermediate transformation matrix M h , for the word embedding matrix M v Decomposition is performed to decouple the dimension of the word vector from the size of the hidden layer, so that the text evaluation model can maintain a smaller word embedding matrix and a larger hidden layer with smaller parameters.

[0070] In some embodiments, the word embedding matrix M v The breakdown is as follows:

[0071] M v =M f ×M h (1)

[0072] Specifically, M v Decompose into M f and M h , M fis the decomposed word embedding matrix, M h is the intermediate conversion matrix, the original word embedding matrix M v The number of parameters is v×m, and the number of parameters after decomposition is v×s+s×m, m>>s, s is the dimension size of the word vector in the decomposed word embedding matrix, M f and M h The parameters of the two matrices are learned as the text evaluation model is trained.

[0073] As a possible implementation of the present application, in the embedding encoding stage of the target text, by querying each word in the sentence S in M f The index in gets the embedded code S emb , embed the code S emb After adding the position code, it is added to the intermediate matrix M h Multiply them together to get the final encoding information X of sentence S, that is:

[0074]

[0075] Perform layer normalization on the matrix X, and the mean μ of X at the corresponding position of the tth word t for:

[0076]

[0077] Standard deviation σ t for:

[0078]

[0079] The normalized value of X at the corresponding position of the tth word is:

[0080]

[0081] Among them, g and b are model parameters, which are obtained through learning during model training. ∈ is a constant. To avoid division by 0, a very small value ∈ is usually added to the denominator. The value of ∈ is usually 1×10 -7 .

[0082] A2: Convert the encoded information according to a preset algorithm to obtain a conversion result of the target text.

[0083] In an embodiment of the present application, the above-mentioned encoded information is converted and processed by the conversion processing layer of the above-mentioned text evaluation model to obtain a conversion processing result of the target text.

[0084] As a possible implementation of the present application, the above step A2 specifically includes:

[0085] A21: Convert the encoded information based on the attention mechanism to obtain an initial conversion result.

[0086] The role of the attention mechanism is to combine the information of other words in the sentence and consider it comprehensively when processing a certain word, so as to effectively include its contextual information.

[0087] In an embodiment of the present application, before the training of the above-mentioned text evaluation model begins, the model parameters are randomly initialized values. When the training of the text evaluation model begins, the model parameters are initialized with random values. The model parameters are continuously iteratively updated as the model is trained. The model parameters obtained after the training is completed will enable the text evaluation model to have good representation capabilities.

[0088] Get the first model parameters of the given text evaluation model, the first model parameters include W q , W k , W y , Among them, q, k, and y are all identifiers.

[0089] In some embodiments, there are c attention heads, and r×c=m, where r is the first model parameter W q 、W k 、W v The size of the second dimension of the model parameter matrix. For the i-th attention head, multiplying the layer-normalized encoded information X of the sentence S with the above three model parameter matrices respectively yields the following results:

[0090] Q=X·W q , K=X·W k , Y=X·W y (6)

[0091] in, The above Q, K, and Y are intermediate variables for calculation.

[0092] Further transformation is performed as follows:

[0093]

[0094]

[0095] in, A is the intermediate variable for calculation, k and j are subscripts, Z i is the intermediate amount of calculation of the i-th attention head, and the calculation results of c attention heads {Z 1 ,Z 2 ,…,Z cPerform matrix concatenation along the second dimension of the model parameter matrix to obtain the initial transformation processing result Z:

[0096]

[0097] A22: Perform residual connection processing on the encoded information and the initial conversion processing result to obtain a residual connection result.

[0098] Specifically, Z and X are residually connected according to the following formula:

[0099]

[0100] X again res Perform layer normalization. The specific calculation process is the same as the above layer normalization, and the result of residual connection can be obtained:

[0101] X norm =layernorm(X res ) (11)

[0102] Layernorm is used here to represent the layer normalization process.

[0103] A23: Performing full connection processing on the residual connection result through a fully connected layer to obtain a conversion processing result of the target text.

[0104] Specifically, in X norm Two fully connected layers are added later. Get the given second model parameters, which include the model parameters W f1 , W f2 ,in d1 represents the matrix W f1 The size of the second dimension, X norm Specifically, the following transformation is performed to obtain the conversion processing result of the sentence in the target text:

[0105] F=relu(X norm W f1 ) (12)

[0106] H 1 =F·W f2 (13)

[0107] Among them, relu(x)=max(x,0), H 1 Represents the output of the Transformer Block conversion layer in layer 1.

[0108] In the embodiment of the present application, the entire text evaluation model has a total of g layers of Transformer Blocks. For example, g can be 12, that is, the text evaluation model has a total of 12 layers of Transformer Blocks. The calculation logic of each layer is the same, and the model parameters are shared between layers. The output H of each Transformer Block layer is converted to o As the input of the next TransformerBlock layer, o is the layer index. The output of the last Transformer Block layer is H g , which is the conversion result of the sentence in the target text,

[0109] In the embodiment of the present application, the model parameters can be greatly reduced through parameter decomposition of the coding layer and parameter sharing of the transformation layer.

[0110] To better utilize the contextual information of each word during model training, a masked language model can be used to learn the semantic representation of each word. Specifically, the words in the original sentence S = {w1,w2,w3,w4,w5,w6} are replaced with a mask [MASK] in a certain ratio. After replacement, S = {w1,w2,M,w4,M,w6}, where M is the mask [MASK]. The model is then asked to predict these masked words. This way, the model predicts the masked words from both the left and right sides of [MASK], allowing it to learn the semantic representation of each word with contextual information.

[0111] Since the word [MASK] does not appear in the real sentence S during prediction, this can cause inconsistencies between the model's training and prediction scenarios. To address this inconsistency, during model training, 80% of the words selected for masking are replaced with [MASK], 10% remain unchanged, and 10% are replaced with other words. This way, during model training, the model does not know which words are masked with [MASK], which are replaced with other words, and which remain unchanged. The model learns context-sensitive semantic representations for each word.

[0112] After training the masked language model, fine-tuning can be performed on it to learn the scores of different evaluation dimensions in the text. In the embodiment of the present application, the text can be scored based on four dimensions: grammatical rationality, richness of vocabulary, correct spelling, and logical coherence.

[0113] A3: Based on the conversion processing result, the probability distribution of the target text at different scoring levels of the plurality of evaluation dimensions is calculated.

[0114] In an embodiment of the present application, the probability distribution is calculated through the output layer of the above-mentioned text evaluation model, and the final scoring result is output.

[0115] As a possible implementation of the present application, the above step A3 specifically includes:

[0116] A31: Taking the mean of the first dimension of the conversion processing result to obtain a mean vector of the conversion processing result.

[0117] Based on the trained mask language model, the output H of the last Transformer Block can be obtained by calculation g ,

[0118] For H g The first dimension of is averaged to obtain the mean vector H of the conversion result. mean ,Right now:

[0119]

[0120] A32: Perform a linear transformation on the mean vector to obtain a linear transformation result of the mean vector.

[0121] Specifically, the linear transformation result O can be calculated according to the following formula:

[0122] O=H mean W o (15)

[0123] in, W o is the model parameter matrix, obtained through learning, and c is the dimension of the final output. In this embodiment, the evaluation is based on four dimensions: grammatical rationality, vocabulary richness, spelling correctness, and logical coherence. Each dimension is divided into three levels: high, medium, and low. Therefore, c = 4 × 3 = 12. A33: Perform a softmax transform on the linear transformation results along different evaluation dimensions to obtain the probability distribution of the target text at different scoring levels along these evaluation dimensions.

[0124] The specific softmax transformation process is as follows:

[0125] p1=softmax(O[1:3])

[0126] p2=softmax(O[4:6])

[0127] p3=softmax(O[7:9])

[0128] p4=softmax(O[10:12]) (16)

[0129] in, represents the probability distribution of the score for grammatical rationality, The probability distribution of the score representing the richness of the vocabulary, represents the probability distribution of the ratings for spelling correctness, The probability distribution of scores representing logical coherence.

[0130] Step S202: For each evaluation dimension, the scoring level with the maximum probability in the probability distribution is used as the scoring result of the target text in the evaluation dimension.

[0131] The specific calculation process is as follows:

[0132]

[0133]

[0134]

[0135]

[0136] Finally, the scoring level of the target text in each evaluation dimension can be obtained.

[0137] In summary, the embodiments of the present application adopt a masking approach, supplemented by technical means such as parameter decomposition and parameter sharing. The language model trained based on a large amount of text data can better learn the semantic representation of each word in the text with contextual information. Further fine-tuning of the trained masked language model on the text data annotated from multiple dimensions is performed, and essays are evaluated from multiple dimensions. This provides effective assistance for teachers in grading essays and students in writing practice, which can improve teachers' grading efficiency and improve students' writing practice results.

[0138] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0139] Corresponding to a text evaluation method described in the above embodiment, Figure 4 A structural diagram of an embodiment of a text evaluation device provided in an embodiment of the present application is shown.

[0140] In this embodiment, a text evaluation device may include:

[0141] A text acquisition unit 401 is used to acquire a target text to be evaluated;

[0142] The text evaluation unit 402 is used to process the target text using a trained text evaluation model to obtain a scoring result for the target text; wherein the text evaluation model is a model obtained by fine-tuning a masked language model based on text data annotated from several preset evaluation dimensions.

[0143] As a possible implementation of the present application, the text evaluation unit may include:

[0144] A probability distribution calculation subunit, configured to calculate the probability distribution of the target text at different scoring levels of the plurality of evaluation dimensions;

[0145] The scoring result determination subunit is used to, for each evaluation dimension, take the scoring level with the maximum probability in the probability distribution as the scoring result of the target text in the evaluation dimension.

[0146] As a possible implementation of the present application, the probability distribution calculation subunit may include:

[0147] A coding information acquisition module, used to obtain the coding information of the target text;

[0148] A conversion processing module, configured to convert the coded information according to a preset algorithm to obtain a conversion processing result of the target text;

[0149] The probability distribution calculation module is used to calculate the probability distribution of the target text at different scoring levels of the multiple evaluation dimensions according to the conversion processing result.

[0150] As a possible implementation of the present application, the probability distribution calculation module may include:

[0151] a mean vector calculation submodule, configured to take the mean of the first dimension of the conversion processing result to obtain the mean vector of the conversion processing result;

[0152] A linear transformation submodule, configured to perform a linear transformation on the mean vector to obtain a linear transformation result of the mean vector;

[0153] The probability distribution calculation submodule is used to perform softmax transformation on the linear transformation results in different evaluation dimensions to obtain the probability distribution of the target text at different scoring levels in the several evaluation dimensions.

[0154] As a possible implementation of the present application, the probability distribution calculation module may include:

[0155] An index acquisition submodule, configured to acquire a word embedding matrix and an index of each word in the target text in a dictionary, wherein the word embedding matrix is ​​a matrix corresponding to the words in the dictionary;

[0156] An embedding code determination submodule, configured to determine the embedding code of the target text based on the word embedding matrix and the index of each word in the target text in the dictionary;

[0157] A position code acquisition submodule is used to acquire the position code of the target text, where the position code is determined based on the relative position information of each word in the target text;

[0158] The coding information determination submodule is used to determine the coding information of the target text according to the embedded coding and the position coding.

[0159] As a possible implementation of the present application, the conversion processing module may include:

[0160] An initial conversion submodule, configured to convert the encoded information based on an attention mechanism to obtain an initial conversion result;

[0161] A residual connection submodule, configured to perform residual connection processing on the encoded information and the initial conversion processing result to obtain a residual connection result;

[0162] The fully connected submodule is used to perform full connection processing on the residual connection result through a fully connected layer to obtain the conversion processing result of the target text.

[0163] As a possible implementation of the present application, the several evaluation dimensions include: grammatical rationality, richness of vocabulary, spelling correctness and logical coherence.

[0164] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0165] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0166] Figure 5 A schematic block diagram of a terminal device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0167] like Figure 5As shown, the terminal device 5 of this embodiment includes: a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50. When the processor 50 executes the computer program 52, the steps in the above-mentioned various text evaluation method embodiments are implemented, such as Figure 1 Alternatively, when the processor 50 executes the computer program 52, the functions of the modules / units in the above-mentioned device embodiments are realized, for example, Figure 4 Functions of modules 401 to 402 are shown.

[0168] Exemplarily, the computer program 52 may be divided into one or more modules / units, which are stored in the memory 51 and executed by the processor 50 to implement the present application. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 52 in the terminal device 5.

[0169] The terminal device 5 can be a computing device such as a desktop computer, a notebook, a palmtop computer, a cloud server, etc. It will be understood by those skilled in the art that Figure 5 It is only an example of the terminal device 5 and does not constitute a limitation on the terminal device 5. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device 5 may also include input and output devices, network access devices, buses, etc.

[0170] The processor 50 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0171] The memory 51 can be an internal storage unit of the terminal device 5, such as a hard disk or memory of the terminal device 5. The memory 51 can also be an external storage device of the terminal device 5, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 5. Furthermore, the memory 51 can also include both an internal storage unit of the terminal device 5 and an external storage device. The memory 51 is used to store the computer program and other programs and data required by the terminal device 5. The memory 51 can also be used to temporarily store data that has been output or is about to be output.

[0172] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0173] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0174] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0175] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0176] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0177] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0178] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable storage media do not include electric carrier signals and telecommunication signals.

[0179] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A text evaluation method, characterized in that: include: Get the target text to be evaluated; The target text is processed using a trained text evaluation model to obtain a scoring result for the target text; wherein the text evaluation model is a model obtained by fine-tuning a masked language model based on text data annotated from a plurality of preset evaluation dimensions, and the text evaluation model includes various Transformer Block layers, model parameters are shared between the Transformer Block layers, and the output of each Transformer Block layer serves as the input of the corresponding next Transformer Block layer; the text evaluation model determines the encoding information of the target text based on a word embedding matrix, and the word embedding matrix is ​​obtained by parameter decomposition based on an intermediate transformation matrix; The target text is processed using the trained text evaluation model to obtain a scoring result of the target text, including: Calculating the probability distribution of the target text at different scoring levels of the plurality of evaluation dimensions; For each evaluation dimension, the rating level with the highest probability in the probability distribution is used as the rating result of the target text in the evaluation dimension; The calculating the probability distribution of the target text at different scoring levels of the plurality of evaluation dimensions includes: Obtaining encoding information of the target text; Converting the encoded information according to a preset algorithm to obtain a conversion result of the target text; According to the conversion processing result, the probability distribution of the target text at different scoring levels of the plurality of evaluation dimensions is calculated.

2. The text evaluation method according to claim 1, characterized in that The step of calculating the probability distribution of the target text at different scoring levels of the plurality of evaluation dimensions according to the conversion processing result includes: Taking the mean of the first dimension of the conversion processing result to obtain a mean vector of the conversion processing result; Performing a linear transformation on the mean vector to obtain a linear transformation result of the mean vector; The linear transformation results are evaluated in different dimensions. Transformation is performed to obtain the probability distribution of the target text at different scoring levels of the multiple evaluation dimensions.

3. The text evaluation method according to claim 1, characterized in that The step of obtaining the encoding information of the target text includes: Obtain the word embedding matrix and the index of each word in the target text in the dictionary, wherein the word embedding matrix is ​​a matrix corresponding to the words in the dictionary; Determining an embedding code of the target text according to the word embedding matrix and the index of each word in the target text in the dictionary; Obtaining a position code of the target text, wherein the position code is determined according to relative position information of each word in the target text; The encoding information of the target text is determined according to the embedded code and the position code.

4. The text evaluation method according to claim 1, characterized in that The converting process of the encoded information according to a preset algorithm to obtain the conversion result of the target text includes: Performing conversion processing on the encoded information based on the attention mechanism to obtain an initial conversion result; Performing residual connection processing on the encoded information and the initial conversion processing result to obtain a residual connection result; The residual connection result is fully connected through a fully connected layer to obtain a conversion processing result of the target text.

5. The text evaluation method according to any one of claims 1 to 4, characterized in that: The evaluation dimensions include: grammatical rationality, richness of vocabulary, spelling correctness and logical coherence.

6. A text evaluation device, characterized in that include: A text acquisition unit, used to acquire a target text to be evaluated; A text evaluation unit is configured to process the target text using a trained text evaluation model to obtain a scoring result for the target text; wherein the text evaluation model is a model obtained by fine-tuning a masked language model based on text data annotated from a plurality of preset evaluation dimensions, and the text evaluation model includes various Transformer Block layers, wherein model parameters are shared between the Transformer Block layers, and the output of each TransformerBlock layer serves as the input of the corresponding next Transformer Block layer; the text evaluation model determines the encoding information of the target text based on a word embedding matrix, wherein the word embedding matrix is ​​obtained by parameter decomposition based on an intermediate transformation matrix; The text evaluation unit comprises: A probability distribution calculation subunit, configured to calculate the probability distribution of the target text at different scoring levels of the plurality of evaluation dimensions; A scoring result determination subunit is configured to, for each evaluation dimension, take the scoring level with the highest probability in the probability distribution as the scoring result of the target text in the evaluation dimension; The probability distribution calculation subunit includes: A coding information acquisition module, used to obtain the coding information of the target text; A conversion processing module, configured to convert the coded information according to a preset algorithm to obtain a conversion processing result of the target text; The probability distribution calculation module is used to calculate the probability distribution of the target text at different scoring levels of the multiple evaluation dimensions according to the conversion processing result.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the text evaluation method according to any one of claims 1 to 5 are implemented.

8. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the text evaluation method according to any one of claims 1 to 5 are implemented.

Images