A controllable generative composition polishing method

Abstract

The present invention discloses a controllable generative composition polishing method. A composition evaluation model with multi-dimensional indicators is used to specifically identify defects in the article and provide solutions, reducing the number of times a complete composition is directly generated, thereby reducing the probability of problems caused by the uncontrollability of natural language generation. A natural language generation model with integrated planning is used to further control the content generated by natural language. Compared with traditional computer-assisted writing technology, the text modified by the generative method conforms to language habits while retaining the diversity of natural language expression, which helps writers to open up new ideas.

Description

Technical Field

[0001] The present invention relates to the field of computer-aided application technology, and in particular to a controllable generative composition polishing method. Background Art

[0002] Natural language generation is the automated process of generating human-readable text from given input information under specific interaction goals. Natural language generation has evolved from the traditional modular generation framework to the current mainstream end-to-end natural language generation framework. Natural language generation has many applications in the field of writing, such as poetry creation, couplets, novel continuation, and auxiliary writing.

[0003] Most of the assisted writing methods in natural language generation rely on topics and keywords to directly generate a complete essay for reference. They have the disadvantages of poor controllability and easy generation of irrelevant and repeated content. Traditional computer-assisted writing methods only provide suggestions on words and phrases, with a single form and limited help. Summary of the invention

[0004] In view of the above-mentioned defects of the prior art, the technical problem to be solved by the present invention is to provide a controllable generative composition polishing method to assist writing, which integrates planning to solve the shortcomings of poor controllability of natural language generation. The generative method has the characteristics of diversity, which solves the problem of the single form of traditional methods and provides more diverse and colorful writing ideas in language form.

[0005] To achieve the above object, the present invention provides a controllable generative composition polishing method, comprising the following steps:

[0006] Step 1: Start writing the essay and enter a narrative essay to be revised.

[0007] Step 2: The composition evaluation model calculates the total score of the composition and the scores of various indicators, that is, the narrative is input into the composition evaluation model to calculate the score, and the composition evaluation model is a linear model;

[0008] Step 3: Determine whether the total score of the composition is greater than the threshold X. The threshold X is set to the maximum score of the data set used in the composition evaluation model training phase. If yes, return to the original composition and the algorithm terminates. Otherwise, jump directly to the next step.

[0009] Step 4: Determine whether the total score of the composition is lower than the threshold Y, where the threshold Y is set to 10% of the threshold X. If so, the generation model based on the article skeleton generates a composition with the original composition as a constraint, and the algorithm terminates after returning the new composition. Otherwise, jump directly to the next step.

[0010] Step 5: Determine whether the index score belonging to the chapter structure is lower than the threshold Z, and the threshold Z is set to 10% of the maximum index score belonging to the chapter structure. If yes, the composition generation model integrating data combination and sorting planning re-plans the composition to generate a narrative text, returns the generated composition and terminates the algorithm, otherwise jumps directly to the next step;

[0011] Step 6: Determine whether the index score belonging to the emotional theme is lower than the threshold A. The threshold A is set to 10% of the maximum index score belonging to the paragraph structure. If yes, the sentence classification model extracts the set of sentences to be modified, and then the sequence-to-sequence model based on the recurrent neural network LSTM generates a set of sentences with rich emotional expressions and replaces the original sentences one by one. Otherwise, jump directly to the next step.

[0012] Step 7, same as step 6, determine whether the score of the indicator belonging to language expression is lower than the set threshold. If yes, generate sentences with rich language expression using a variety of rhetorical techniques and replace the original sentences one by one in the same way. Otherwise, jump directly to the next step;

[0013] Step 8: Input the composition into the Transformers-based sentence correction model to correct semantic and grammatical errors;

[0014] Step 9: End, terminate the algorithm, and return the modified composition.

[0015] Furthermore, the steps of the composition evaluation model in step 1 in the training stage and the application stage are respectively:

[0016] During the training phase:

[0017] Step 1-1: Preprocess the corpus of essays containing manual ratings, and extract the feature vector x = (x1, x2, ..., x n ), corresponding to the manual score y of this essay;

[0018] Step 1-2: Substitute the processed data into the linear model f(x;w)=w T x;

[0019] Step 1-3: Use the gradient descent method to estimate the parameters, first initialize w = 0, and then iterate through the following formula:

[0020] w←w+αX(yX T ω)

[0021] During the application phase:

[0022] Step 1-4: Perform the same data preprocessing as in the training phase on the input essay to obtain the feature vector X = (x1, x2, ..., x n );

[0023] Step 1-5: Divide the detailed scoring indicators into three major categories: emotional theme, language expression, and text structure. Also use the linear model f(x) = w T x, you can get the score of each indicator;

[0024] Step 1-6: Use the parameters W obtained in the training phase = (w1, w2..., w n ) Calculate the total score of the composition and the score of each indicator respectively.

[0025] Furthermore, the specific model training algorithm steps of the generation model of the article skeleton in step 4 are as follows:

[0026] Step 4-1: Randomly initialize the sequence generation model G φ And the skeleton extraction model E γ The weight parameters φ, γ, where G φ Contains input to the skeleton module Q α and skeleton to sentence module D θ ;Q α , D θ Both are sequence-to-sequence models, and their encoders and decoders are single-layer LSTM recurrent neural network structures with attention mechanisms;

[0027] Step 4-2: Pre-train E on the sentence compression task γ , input the original corpus x and its compressed corpus s = {s1, ..., s i , ..., s T}, and the cross entropy loss function is used for training, as shown in formula (1):

[0028]

[0029] Step 4-3: E γ Generate a sentence skeleton s j ;

[0030] Step 4-4: Given s j , calculate G φ The loss function is, where the input constraint c and the skeleton s = {s1, ..., s i , ..., s T}, Q α The cross entropy loss function used is shown in formula (2):

[0031]

[0032] Input skeleton s and sentence y = {y1, ..., y i , ..., y M}, Dθ The cross entropy loss function used is shown in formula (3):

[0033]

[0034] Step 4-5: Calculate the reward function R c , where E γ Calculation of R by REINFORCE method c To optimize the parameters, R c It can be expressed as:

[0035]

[0036] Step 4-6: Calculate E γ The gradient function is shown in formula (5):

[0037]

[0038] Step 4-7: Update the model parameter γ. If the stopping condition is met during iteration, the algorithm stops; otherwise, return to step 3.

[0039] Furthermore, the steps of the composition generation model in step 5 to re-plan the composition to generate a narrative text are as follows:

[0040] Step 5-1: Narrative input;

[0041] Step 5-2: Extract the information of input data based on knowledge graph theory, and encode it into an abstract data set by the data encoder

[0042] Step 5-3: The planning process begins. The planning decoder combines and sorts the data, specifying the abstract data to be included for each sentence generated in the text generation phase. This process can be formally expressed as

[0043]

[0044] in, is a sequence of information groups, each of which It is abstract data A subset of each d i yes An element of represents an abstract information, which determines the sentence Y j What needs to be included;

[0045] Step 5-4: Use the greedy algorithm to solve the plan. The plan decoder calculates each input element belongs to the current group Probability As shown in formula (2), Consists of all input elements with probability greater than 0.5;

[0046]

[0047] Among them, {W p , U p , b p} is a trainable parameter, h i It is an abstract element i The encoding vector representation of s j is the hidden state vector of the planning decoder;

[0048] Step 5-5: Encoding as a vector and serves as the input of the planning decoder for the next time step, It is expressed as follows using formula (3):

[0049]

[0050] Step 5-6: Calculate the probability that the planning decoder stops the planning process at the next step

[0051]

[0052] Among them, {W s , b s} is a trainable parameter;

[0053] Step 5-7: If The planning process ends and the information group sequence is obtained. Otherwise, return to step 3;

[0054] Step 5-8: Group the information and its encoding vector Input word decoder decodes sentence Y j ;

[0055] Step 5-9: Decode sentence Y using the word decoder j The hidden state vector generated in the process serves as the input of the word decoder for the next time step;

[0056] Step 5-10: If all the information groups in the sequence have been processed, the final sentence sequence Y=(Y1, Y2, ..., Y M ), otherwise, return to step 8; at the same time, for the trainable parameters in the model, the available loss function Planning decoder and word decoder in supervised training model, They are expressed by formula (5) and formula (6) respectively:

[0057]

[0058]

[0059] Furthermore, the sentence classification model design method for identifying emotion-expressing sentences in step 6 is as follows:

[0060] In the training phase, the training set containing emotion-expressing sentences and other sentences is preprocessed and input into the BERT text classification model based on the pre-trained model. The weight parameters of the fully connected layer of the neural network are fine-tuned by the gradient descent method to minimize the cross entropy loss function.

[0061] In the application stage, the set of sentences to be classified, after the same preprocessing, is input into the BERT text classification model with trained parameters to obtain a set of sentences that belong to emotional expression; the obtained set of sentences is input into the sequence-to-sequence model based on the recurrent neural network LSTM to generate a set of sentences that belong to emotional expression composed of a variety of lyrical techniques, and replace the original sentences one by one.

[0062] Furthermore, each composition is scored according to indicators including but not limited to: the number of words with emotional connotations, the number of times the emotional connotations change, the ratio of words with positive to negative emotional connotations, the fluency score calculated by the language model, paragraph latent semantic analysis, the total number of words, the average paragraph length, the number of sentences containing metaphors, parallelism and other rhetorical devices, the number of idioms, famous sayings, beautiful vocabulary and beautiful sentences.

[0063] The beneficial effects of the present invention are:

[0064] (1) The essay evaluation model uses a multi-dimensional indicator to identify the defects of the essay and provide solutions, reducing the number of times a complete essay is directly generated, thereby reducing the probability of problems caused by the uncontrollable nature of natural language generation.

[0065] (2) A natural language generation model with fusion planning is adopted to further control the content of natural language generation.

[0066] (3) Compared with traditional computer-assisted writing technology, the text modified by the generative method conforms to language habits while retaining the diversity of natural language expression, helping writers to expand their thinking.

[0067] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 is a flow chart of the method of the present invention;

[0069] Figure 2 The flowchart of the algorithm of the generation model based on the article skeleton of the present invention;

[0070] Figure 3 This is a flow chart of the algorithm of the composition generation model that integrates data combination and sorting planning of the present invention. DETAILED DESCRIPTION

[0071] like Figure 1-3 As shown, the present invention provides a controllable generative composition polishing method, and the specific implementation examples are as follows:

[0072] (1) Start writing the essay and enter a narrative essay to be revised;

[0073] (2) The composition evaluation model calculates the total score of the composition and the scores of various indicators. That is, the narrative is input into the composition evaluation model to calculate the score. The composition evaluation model is a linear model. The model training and application steps are as follows:

[0074] During the training phase:

[0075] Step 1: Preprocess the corpus of essays with manual scoring, and extract the feature vector X = (x1, x2, ..., x3) from each essay according to the scoring indicators (the number of words with emotional color, the number of changes in emotional color, the ratio of positive and negative emotional color words, the fluency score calculated by the language model, paragraph latent semantic analysis, the total number of words, the average paragraph length, the number of sentences containing metaphors, parallelism and other rhetorical devices, the number of beautiful words and beautiful sentences such as idioms and famous sayings, etc.). n ), corresponding to the manual score y of this essay;

[0076] Step 2: Substitute the processed data into the linear model f(x;w)=w T x;

[0077] Step 3: Use the gradient descent method to estimate the parameters. Initialize w = 0, and then iterate using the following formula:

[0078] w←ω+αX(yX T ω) (1)

[0079] During the application phase:

[0080] Step 1: Perform the same data preprocessing as in the training phase on the input essay to obtain the feature vector x = (x1, x2, ..., x n );

[0081] Step 2: Divide the detailed scoring indicators into three major categories: emotional theme, language expression, and text structure. Also use the linear model f(x) = w T x, you can get the score of each indicator;

[0082] Step 3: Use the parameters W obtained in the training phase = (w1, w2..., w n ) Calculate the total score of the composition and the score of each indicator respectively.

[0083] (3) Determine whether the total score of the composition is greater than the threshold X. The threshold X is set to the maximum score of the data set used in the composition evaluation model training phase. If so, return to the original composition and the algorithm terminates. Otherwise, jump directly to the next step.

[0084] (4) Determine whether the total score of the composition is lower than the threshold Y, which is set to 10% of the threshold X. If it is, the generation model based on the article skeleton generates a composition with the original composition as the constraint condition, and the algorithm terminates after returning the new composition. Otherwise, it jumps directly to the next step. If the total score of the composition is too low, it means that the input composition is incomplete, incomplete in structure, or too simple in content. In this case, local modification is no longer effective, so the original composition is used as the constraint condition, and a natural language generation method based on planning is used to generate a composition with a complete and standardized structure for reference. Taking the composition of narrative as an example, the algorithm design idea of ​​the generation model based on the article skeleton is as follows: The writing of narrative generally unfolds according to certain clues. For example, when narrating, it unfolds according to the order of events. The six elements of narrative are time, place, characters, cause of event, process, and result. In summary, a narrative generation task based on planning can be proposed: given a constraint set c, it is required to generate a narrative Y that meets the constraints. The constraint set c can be the outline of the event, clues to the development of the event such as character clues, or the initial state, intermediate state, and final state of the event. The neural language generation model based on the encoder-decoder framework can quickly establish the connection between the constraint set c and the narrative Y under data-driven conditions, that is, starting from a given constraint set, a certain intermediate representation of the plot of the article is planned, such as an abstract sequence of events, a general development clue (skeleton) of the event, etc., and then a narrative is generated based on this intermediate representation. For example, assuming that the given constraint is the beginning of the composition, and the task is to generate subsequent plots, such as constraint c = "That was an old man selling vegetables, wearing a straw hat, a pair of muddy slippers, dragging a shabby bicycle with two bamboo baskets of vegetables tied to it, and hawking at the door of my house. His hoarse voice seemed to have exhausted all his strength, just to attract people to come and patronize his business.", then the model can learn the potential event clues of the old man hawking hard, and the automatically predicted clues for the following content may be "he strongly recommends his own dishes after meeting people", and based on the clues, it can further generate "Every time he passes by someone, he will stop for a while and look into the house with expectant eyes to see if anyone comes out to buy his vegetables. As long as he sees someone coming out, he comes up to recommend all kinds of dishes, and he wants people to buy all his remaining vegetables." The specific model training algorithm steps are as follows:

[0085] Step 1: Randomly initialize the sequence generation model G φ And the skeleton extraction model E γ The weight parameters φ, γ. Among them, G φ Contains input to the skeleton module Q α and skeleton to sentence module D θ .Q α , D θ Both are sequence-to-sequence models, and their encoders and decoders are single-layer LSTM recurrent neural network structures with attention mechanisms.

[0086] Step 2: Pre-train E on the sentence compression task γ , input the original corpus x and its compressed corpus s = {s1, ..., s i , ..., s T}, and the cross entropy loss function is used for training, as shown in formula (1).

[0087]

[0088] Step 3: E γ Generate a sentence skeleton s j .

[0089] Step 4: Given s j , calculate G φ The loss function is: i , ..., s T}, Q α The cross entropy loss function used is shown in formula (2).

[0090]

[0091] Input skeleton s and sentence y = {y1, ..., y i , ..., y M}, D θ The cross entropy loss function used is shown in formula (3).

[0092]

[0093] Step 5: Calculate the reward function R c Among them, E γ Calculation of R by REINFORCE method c To optimize the parameters, R c It is expressed as formula (4).

[0094]

[0095] Step 6: Calculate E γThe gradient function of is shown in formula (5).

[0096]

[0097] Step 7: Update the model parameter γ. If the stopping condition is met during the iteration, the algorithm stops; otherwise, return to step 3.

[0098] (5) Determine whether the index score belonging to the chapter structure is lower than the threshold Z. The threshold Z is set to 10% of the maximum index score belonging to the chapter structure. If it is, the composition generation model that integrates data combination and sorting planning will re-plan and generate the composition, return the generated composition and terminate the algorithm, otherwise jump directly to the next step. When the input narrative scores too low in terms of chapter structure, there are often problems such as inappropriate details and poor overall coherence. Generally speaking, people will prepare an outline before writing an article, and then expand each part of the outline in detail when writing the article. The planning mechanism based on data combination and sorting draws on this process, plans and grasps the content of the article as a whole, and can control the content of each sentence in the narrative, the length of each part in the outline, and the overall structure of the article. Specifically, the process of model re-planning to generate narratives is as follows:

[0099] Step 1: Narrative input;

[0100] Step 2: Extract the information of input data based on knowledge graph theory, and encode it into an abstract data set by the data encoder

[0101] Step 3: The planning process begins. The planning decoder combines and sorts the data, specifying the abstract data to be included for each sentence generated in the text generation phase. This process can be formally expressed as

[0102]

[0103] in, is a sequence of information groups, each of which It is abstract data A subset of each d i yes An element of represents an abstract information, which determines the sentence Y j What needs to be included;

[0104] Step 4: Use the greedy algorithm to solve the plan. The plan decoder calculates each input element belongs to the current group Probability As shown in formula (2). Consists of all input elements with probability greater than 0.5.

[0105]

[0106] Among them, {W p , U p , b p} is a trainable parameter, h i It is an abstract element i The encoding vector representation of s j is the hidden state vector of the planning decoder;

[0107] Step 5: Encoding as a vector And serves as the input of the planning decoder for the next time step. It is expressed as follows using formula (3):

[0108]

[0109] Step 6: Calculate the probability that the planning decoder stops the planning process at the next step

[0110]

[0111] Among them, {W s , b s} is a trainable parameter;

[0112] Step 7: If The planning process ends and the information group sequence is obtained Otherwise, return to step 3;

[0113] Step 8: Group the information and its encoding vector Input word decoder decodes sentence Y j ;

[0114] Step 9: Decode sentence Y using the word decoder j The hidden state vector generated in the process serves as the input of the word decoder for the next time step;

[0115] Step 10: If all the information groups in the sequence have been processed, the final sentence sequence Y=(Y1, Y2, ..., Y M ), otherwise, return to step 8.

[0116] Regarding the trainable parameters in the model, the available loss function Planning decoder and word decoder in supervised training model, They are expressed by formula (5) and formula (6) respectively.

[0117]

[0118]

[0119] (6) Determine whether the score of the index belonging to the emotional theme is lower than the threshold A. The threshold A is set to 10% of the maximum score of the index belonging to the chapter structure. If yes, the sentence classification model extracts the set of sentences to be modified, and then generates a set of sentences rich in emotional expression by the sequence-to-sequence model based on the recurrent neural network LSTM, and replaces the original sentences one by one. Otherwise, jump directly to the next step. That is, traverse all sentences in the narrative, and identify sentences that may belong to emotional expression through the sentence classification model. The design idea of ​​the sentence classification model for identifying emotional expression sentences is as follows: in the training stage, the training set containing emotional expression sentences and other sentences is preprocessed and input into the BERT text classification model based on the pre-trained model. The weight parameters of the fully connected layer of the neural network are fine-tuned by the gradient descent method to minimize the cross entropy loss function; in the application stage, the set of sentences to be classified is preprocessed in the same way and input into the BERT text classification model with trained parameters to obtain a set of sentences that belong to emotional expression. The obtained sentence set is input into the sequence-to-sequence model based on the recurrent neural network LSTM to generate a sentence set belonging to emotional expression composed of various lyrical techniques (using scenery to express emotions, being moved by scenery, expressing emotions through objects, expressing aspirations through objects, expressing one's feelings directly, integrating emotions into events, and integrating emotions into reason), and replace the original sentences one by one.

[0120] (7) Same as step (6), determine whether the score of the indicator belonging to language expression is lower than the set threshold. If yes, generate sentences with rich language expression using a variety of rhetorical techniques and replace the original sentences one by one according to the same method; otherwise, jump directly to the next step;

[0121] (8) Input the essay into the Transformers-based sentence correction model to correct semantic and grammatical errors;

[0122] (9) End, terminate the algorithm, and return the modified composition.

[0123] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A controllable generative composition polishing method, characterized in that: The following steps are involved: Step 1: Start writing the essay and enter a narrative essay to be revised. Step 2: The composition evaluation model calculates the total score of the composition and the scores of various indicators, that is, the narrative is input into the composition evaluation model to calculate the score, and the composition evaluation model is a linear model; Step 3: Determine whether the total score of the composition is greater than the threshold X. The threshold X is set to the maximum score of the data set used in the composition evaluation model training phase. If yes, return to the original composition and the algorithm terminates. Otherwise, jump directly to the next step. Step 4: Determine whether the total score of the composition is lower than the threshold Y, where the threshold Y is set to 10% of the threshold X. If so, the generation model based on the article skeleton generates a composition with the original composition as a constraint, and the algorithm terminates after returning the new composition. Otherwise, jump directly to the next step. Step 5: Determine whether the index score belonging to the chapter structure is lower than the threshold Z, and the threshold Z is set to 10% of the maximum index score belonging to the chapter structure. If yes, the composition generation model integrating data combination and sorting planning re-plans the composition to generate a narrative text, returns the generated composition and terminates the algorithm, otherwise jumps directly to the next step; Step 6: Determine whether the index score belonging to the emotional theme is lower than the threshold A. The threshold A is set to 10% of the maximum index score belonging to the paragraph structure. If yes, the sentence classification model extracts the set of sentences to be modified, and then the sequence-to-sequence model based on the recurrent neural network LSTM generates a set of sentences with rich emotional expressions and replaces the original sentences one by one. Otherwise, jump directly to the next step. Step 7, same as step 6, determine whether the score of the indicator belonging to language expression is lower than the set threshold. If yes, generate sentences with rich language expression using a variety of rhetorical techniques and replace the original sentences one by one in the same way. Otherwise, jump directly to the next step; Step 8: Input the composition into the Transformers-based sentence correction model to correct semantic and grammatical errors; Step 9: End, terminate the algorithm, and return the modified composition; The specific model training algorithm steps of the generation model based on the article skeleton in step 4 are as follows: Step 4-1: Randomly initialize the sequence generation model G φ And the skeleton extraction model E γ The weight parameters φ, γ, where G φ Contains input to the skeleton module Q α and skeleton to sentence module D θ ;Q α , D θ Both are sequence-to-sequence models, and their encoders and decoders are single-layer LSTM recurrent neural network structures with attention mechanisms; Step 4-2: Pre-train E on the sentence compression task γ , input the original corpus x and its compressed corpus s′=(s′1,s′2,...,s′ T ), and the cross entropy loss function is used for training, as shown in formula (1): Step 4-3: E γ Generate a sentence skeleton s j ; Step 4-4: Given a skeleton s j , calculate G φ The loss function is: n ), Q α The cross entropy loss function used is shown in formula (2): Input skeleton sequence s and sentence y = (y1, y2, ..., y m ), D θ The cross entropy loss function used is shown in formula (3): Step 4-5: Calculate the reward function R c , where E γ Calculation of R by REINFORCE method c To optimize the parameters, R c It can be expressed as formula (4): Step 4-6: Calculate E γ The gradient function is shown in formula (5): Step 4-7: Update the model parameter γ. If the stop condition is met during iteration, the algorithm stops; otherwise, return to step 3; The steps of the composition generation model in step 5 to re-plan the composition to generate a narrative text are as follows: Step 5-1: Narrative input; Step 5-2: Extract the information of input data based on knowledge graph theory, and encode it into an abstract data set by the data encoder Step 5-3: The planning process begins. The planning decoder combines and sorts the data, specifying the abstract data to be included for each sentence generated in the text generation phase. This process can be formally expressed as: Where G = (G1, G2, ..., G M ) is a sequence of information groups, each information group G j =(d1, d2, ..., d n ) is abstract data A subset of each d i yes An element of represents an abstract information, which determines the sentence Y j What needs to be included; Step 5-4: Use the greedy algorithm to solve the plan. The plan decoder calculates each input element belongs to the current group G j The probability P(d i ∈G j |G ＜j , A), as shown in formula (7), G j Consists of all input elements with probability greater than 0.5; Among them, {W p , U p , b p } is a trainable parameter, h i It is an abstract element i The encoding vector representation of is the hidden state vector of the planning decoder; Step 5-4: G j The encoding is represented as a vector e(G j ) and serves as the input of the planning decoder for the next time step, e(G j ) is expressed as follows using formula (8): Step 5-6: Calculate the probability that the planning decoder stops the planning process at the next step Among them, {W s , b s } is a trainable parameter; Step 5-7: If The planning process ends, and the information group sequence G=(G1, G2, ..., G M ), otherwise, return to step 3; Step 5-8: Group G j and its encoding vector e(G j ) Input word decoder decodes sentence Y j ; Step 5-9: Decode sentence Y using the word decoder j The hidden state vector generated in the process serves as the input of the word decoder for the next time step; Step 5-10: If all the information groups in the sequence have been processed, the final sentence sequence Y=(Y1, Y2, ..., Y M ), otherwise, return to step 8; at the same time, for the trainable parameters in the model, the available loss function Planning decoder and word decoder in L2 supervised training model, They are expressed by formula (10) and formula (11) respectively:

2. A controllable generative composition polishing method as claimed in claim 1, characterized in that: The steps of the composition evaluation model in step 1 in the training stage and the application stage are as follows: During the training phase: Step 1-1: Preprocess the corpus of essays containing manual ratings, and extract the feature vector X = (x1, x2, ..., x n ), corresponding to the manual rating y of this work; Step 1-2: Substitute the processed data into the linear model f(x;w)=w T x; Step 1-3: Use the gradient descent method to estimate the parameters, first initialize w = 0, and then iterate through the following formula: w←w+αx(y-X T w) During the application phase: Step 1-4: Perform the same data preprocessing as in the training phase on the input essay to obtain the feature vector X = (x1, x2, ..., x n ); Step 1-5: Divide the detailed scoring indicators into three major categories: emotional theme, language expression, and text structure. Also use the linear model f(x; w) = w T x, get the score of each indicator; Step 1-6: Use the parameters W obtained in the training phase = (w1, w2, ..., w n ) Calculate the total score of the composition and the score of each indicator respectively.

3. A controllable generative composition polishing method as claimed in claim 1, characterized in that: The sentence classification model design method for identifying emotion-expressing sentences in step 6 is as follows: In the training phase, the training set containing emotion-expressing sentences and other sentences is preprocessed and input into the BERT text classification model based on the pre-trained model. The weight parameters of the fully connected layer of the neural network are fine-tuned by the gradient descent method to minimize the cross entropy loss function. In the application stage, the set of sentences to be classified, after the same preprocessing, is input into the BERT text classification model with trained parameters to obtain a set of sentences that belong to emotional expression; the obtained set of sentences is input into the sequence-to-sequence model based on the recurrent neural network LSTM to generate a set of sentences that belong to emotional expression composed of a variety of lyrical techniques, and replace the original sentences one by one.

4. A controllable generative composition polishing method as claimed in claim 2, characterized in that: Each essay is scored according to the following indicators: the number of words with emotional connotations, the number of times the emotional connotations change, the ratio of words with positive to negative emotional connotations, the fluency score calculated by the language model, the latent semantic analysis of the paragraphs, the total number of words, the average paragraph length, the number of sentences containing metaphors and parallelism, the number of idioms, famous sayings, beautiful vocabulary and beautiful sentences.

Images