Text generation diversity control method and device, equipment and storage medium

Abstract

The invention belongs to the technical field of text rewriting, and particularly provides a diversity control method and device for text generation, equipment and a storage medium, and the method comprises the following steps: preprocessing an input text to generate a word sequence; syntactic component analysis is conducted on the text, component types are divided according to analysis results, and type results are marked on all words; setting a penalty coefficient of each component type; calculating the probability distribution of the next word by using a text generation model, and adjusting the probability according to a penalty coefficient corresponding to the component type of the word; and generating a next character or word by adopting a text generation algorithm according to the adjusted probability distribution until the whole text rewriting is completed. By setting the adjustable penalty coefficient, the text generation model can be flexibly adjusted according to different business scenes or user requirements. And the applicability of the model is improved.

Description

Background Art

[0002] In some specific fields, it may be required that the content generated by the text rewriting model has a high degree of diversity, that is, the difference from the input text is as large as possible on the premise of ensuring the same semantics.

[0003] The usual approach is to impose a penalty on the words that have already appeared in the input text during the process of generating the input text: that is, set a penalty coefficient k (0 < k < 1). When the text generation model calculates the probability distribution of the next word, multiply the probabilities of those words that have appeared in the input text by k to reduce their probabilities of appearing in the output text, and keep the probabilities of those words that have not appeared in the input text unchanged. The smaller k is set, the greater the diversity of the generated text. However, too small k may also cause the semantics of the generated text to deviate from the input text, thus violating the premise of the text rewriting model of "keeping the semantics basically unchanged". Moreover, if the probability distribution calculated by the model is disturbed too much, it may cause the output text to be chaotic (such as grammar errors, logical confusion, expression errors, etc.). Summary of the Invention

[0004] In order to solve the technical problems that in the existing text rewriting, it is difficult to effectively balance the contradiction between semantic preservation and diversity enhancement while improving the diversity of the generated content, and to avoid semantic deviation or quality degradation of the output text caused by improper setting of the penalty coefficient, the present invention provides a diversity control method, device, equipment and storage medium for text generation.

[0005] In a first aspect, the technical solution of the present invention provides a diversity control method for text generation, including the following steps: Preprocess the input text to generate a word sequence; Perform syntactic component analysis on the text, divide the component types according to the analysis results, and mark the type results on each word; Set the penalty coefficient for each component type; Use the text generation model to calculate the probability distribution of the next word and adjust the probability according to the penalty coefficient corresponding to the component type of the word; Generate the next word or character according to the adjusted probability distribution using the text generation algorithm until the entire text rewriting is completed.

[0006] As a further limitation of the technical solution of the present invention, the step of preprocessing the input text to generate a word sequence includes: Receive the original text input by the user; Use a word segmentation tool to perform word segmentation on the text to obtain a word sequence.

[0007] As a further limitation of the technical solution of the present invention, the steps of performing syntactic component analysis on the text, dividing the component types according to the analysis results and marking the type results on each word include: Apply a syntactic analyzer to parse the text and identify the syntactic components of each word; The identified syntactic components are divided into component types, namely, main components and modifying components; The corresponding component type is marked on each word to form a word sequence with syntactic component type labels.

[0008] As a further limitation of the technical solution of the present invention, the step of setting the penalty coefficient for each component type includes: Set the penalty coefficient k1 of the main component and the penalty coefficient k2 of the modified component; the value range of k1 and k2 are both between 0 and 1, and k1>k2.

[0009] As a further limitation of the technical solution of the present invention, the steps of using the text generation model to calculate the probability distribution of the next word and adjusting the probability according to the penalty coefficient corresponding to the component type of the word include: Use the text generation model to calculate the probability distribution of the next word; For each word, check its syntactic component type label; If it is a main component, multiply its probability by the penalty coefficient of the main component to obtain the adjusted probability distribution; If it is a modification component, multiply its probability by the penalty coefficient of the modification component to obtain the adjusted probability distribution; If it is not marked, multiply its probability by 1 to get the adjusted probability distribution.

[0010] As a further limitation of the technical solution of the present invention, the step of generating the next character or word by using a text generation algorithm according to the adjusted probability distribution until the entire text rewriting is completed includes: Generate the next character or word using a text generation algorithm according to the adjusted probability distribution; Add the newly generated characters or words to the generated text sequence until the entire text is rewritten; Organize the generated text; Output the collated text to a specified location or format.

[0011] As a further limitation of the technical solution of the present invention, the step of arranging the generated text includes: Remove extra spaces and punctuation marks from the generated text; Format or format the text as desired.

[0012] In a second aspect, the technical solution of the present invention provides a diversity control device for text generation, including a preprocessing module, a component analysis module, a penalty coefficient setting module, a probability adjustment module and a generation processing module; A preprocessing module is used to preprocess the input text to generate a word sequence; The component analysis module is used to perform syntactic component analysis on the text, divide the component types according to the analysis results and mark the type results on each word; The penalty coefficient setting module is used to set the penalty coefficient of each component type; A probability adjustment module, used to calculate the probability distribution of the next word using the text generation model and adjust the probability according to the penalty coefficient corresponding to the component type of the word; The generation processing module is used to generate the next character or word using a text generation algorithm according to the adjusted probability distribution until the entire text is rewritten.

[0013] As a further limitation of the technical solution of the present invention, the preprocessing module includes a receiving unit and a word segmentation processing unit; A receiving unit, used for receiving original text input by a user; The word segmentation processing unit is used to use the word segmentation tool to perform word segmentation on the text to obtain a word sequence.

[0014] As a further limitation of the technical solution of the present invention, the component analysis module includes an analysis unit, a division unit and a label setting unit; An analysis unit, for applying a syntactic analyzer to parse the text and identify the syntactic components of each word; A division unit is used to divide the identified syntactic components into component types, namely, main components and modifying components; The label setting unit is used to mark the corresponding component type on each word to form a word sequence with syntactic component type labels.

[0015] As a further limitation of the technical solution of the present invention, a penalty coefficient setting module sets a penalty coefficient k1 for the main component and a penalty coefficient k2 for the modified component; wherein the value ranges of k1 and k2 are both between 0 and 1, and k1>k2.

[0016] As a further limitation of the technical solution of the present invention, the probability adjustment module includes an execution unit, a label checking unit and an adjustment processing unit; An execution unit, for calculating the probability distribution of the next word using a text generation model; A label checking unit, used for checking the syntactic component type label of each word; The adjustment processing unit is used to, if it is a main component, multiply its probability by the penalty coefficient of the main component to obtain an adjusted probability distribution; if it is a modified component, multiply its probability by the penalty coefficient of the modified component to obtain an adjusted probability distribution; if it is unmarked, multiply its probability by 1 to obtain an adjusted probability distribution.

[0017] As a further limitation of the technical solution of the present invention, the generation processing module includes a calculation unit, an addition processing unit, a text sorting unit and an output unit; A computing unit, used for generating the next character or word by using a text generation algorithm according to the adjusted probability distribution; An adding processing unit, used for adding the newly generated characters or words to the generated text sequence until the entire text is rewritten; A text collating unit, used for collating the generated text; The output unit is used to output the sorted text to a specified location or format.

[0018] As a further limitation of the technical solution of the present invention, the text sorting unit is specifically used to remove redundant spaces and punctuation marks from the generated text; and to format or typeset the text as needed.

[0019] In a third aspect, the technical solution of the present invention also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores computer program instructions executable by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the diversity control method for text generation as described in the first aspect.

[0020] In a fourth aspect, the technical solution of the present invention further provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the diversity control method for text generation as described in the first aspect.

[0021] It can be seen from the above technical solutions that the present invention has the following advantages: (1) Improving text diversity without losing semantic accuracy: The present invention effectively solves the semantic deviation problem that is easily caused by traditional methods when pursuing text diversity by finely distinguishing the syntactic components in the text and implementing different penalty strategies for different components. While ensuring the stable output of key information (such as subject, predicate, object, etc.), the diversity of secondary components (such as modifiers, conjunctions, etc.) is increased, so that the generated text not only retains the core meaning of the original text, but also shows a rich form of expression.

[0022] (2) Enhance the flexibility and adaptability of the text generation model: By setting adjustable penalty coefficients (k1 and k2), the present invention enables the text generation model to be flexibly adjusted according to different business scenarios or user needs. This flexibility not only improves the applicability of the model, but also allows developers or users to optimize the generation effect by fine-tuning parameters to meet the specific requirements of different fields.

[0023] (3) Reduce the error rate and cost of text generation: By reducing grammatical errors, logical confusion and other problems caused by excessive penalties, the present invention reduces the error rate of text generation and improves the quality of generated text. At the same time, since the generation effect can be automatically optimized to a certain extent, the need for manual editing and modification is reduced, thereby reducing the cost and time cost of text generation.

[0024] In addition, the invention has a reliable design principle, a simple structure and a very broad application prospect. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0026] Figure 1 is a schematic flow chart of a method according to an embodiment of the present invention.

[0027] Figure 2 is a schematic block diagram of an apparatus according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] like Figure 1 As shown, an embodiment of the present invention provides a diversity control method for text generation, comprising the following steps: Step 1: Preprocess the input text to generate word sequences; Step 2: Analyze the text into syntactic components, divide the components into types according to the analysis results and mark the type results on each word; Step 3: Set the penalty coefficient for each component type; Step 4: Use the text generation model to calculate the probability distribution of the next word and adjust the probability according to the penalty coefficient corresponding to the component type of the word; Step 5: Generate the next character or word using a text generation algorithm based on the adjusted probability distribution until the entire text is rewritten.

[0030] In some embodiments, the step of preprocessing the input text to generate a word sequence includes: Receive raw text input from the user; Use the word segmentation tool to segment the text and obtain a word sequence.

[0031] In some embodiments, the steps of performing syntactic component analysis on the text, dividing the component types according to the analysis results and marking the type results on each word include: Apply a syntactic analyzer to parse the text and identify the syntactic components of each word; The identified syntactic components are divided into component types, namely, main components and modifying components; For example, subject, predicate, object, time and place, and name of a person are the main components, while complement, attributive, adverbial, conjunction, and function words are the modifying components.

[0032] The corresponding component type is labeled on each word to form a word sequence with syntactic component type labels.

[0033] In some embodiments, the step of setting the penalty coefficient for each component type includes: The penalty coefficient k1 of the main component and the penalty coefficient k2 of the modified component are set; wherein the value ranges of k1 and k2 are both between 0 and 1, and k1>k2. In the embodiment of the present invention, the value of k1 is 0.95 and the value of k2 is 0.7.

[0034] In some embodiments, the steps of using the text generation model to calculate the probability distribution of the next word and adjusting the probability according to the penalty coefficient corresponding to the component type of the word include: The probability distribution of the next word is calculated using a text generation model; in the embodiment of the present invention, the document generation model can be a trained model such as one based on RNN, LSTM or Transformer.

[0035] For each word, check its syntactic component type label; If it is a main component, multiply its probability by the penalty coefficient of the main component to obtain the adjusted probability distribution; If it is a modification component, multiply its probability by the penalty coefficient of the modification component to obtain the adjusted probability distribution; If it is not marked, multiply its probability by 1 to get the adjusted probability distribution.

[0036] In some embodiments, the step of generating the next character or word using a text generation algorithm according to the adjusted probability distribution until the entire text rewriting is completed includes: Generate the next word or phrase using a text generation algorithm based on the adjusted probability distribution; input the initial text or text fragment into the model. Initialize the model's hidden state (for sequence models such as RNN and LSTM) or encoder-decoder state (for Seq2Seq models). The model calculates the probability distribution of the next word or phrase based on the current input (including the generated text and possible context information). This probability distribution represents the probability of each candidate word or phrase appearing in the model.

[0037] Add the newly generated word or phrase to the generated text sequence until the entire text is rewritten; sample a word or phrase from the adjusted probability distribution as the next output. Update the model's hidden state or encoder-decoder state to reflect the newly generated text information.

[0038] The sampling method can be a simple greedy algorithm (selecting the word or term with the highest probability) or a more complex algorithm such as temperature-based sampling (softmax temperature sampling), top-k sampling, nucleus sampling, etc., to increase the diversity of generation.

[0039] Organize the generated text; Output the collated text to a specified location or format.

[0040] In some embodiments, the step of collating the generated text includes: Remove extra spaces and punctuation marks from the generated text; Format or format the text as desired.

[0041] like Figure 2 As shown, an embodiment of the present invention provides a diversity control device for text generation, including a preprocessing module, a component analysis module, a penalty coefficient setting module, a probability adjustment module and a generation processing module; A preprocessing module is used to preprocess the input text to generate a word sequence; The component analysis module is used to perform syntactic component analysis on the text, divide the component types according to the analysis results and mark the type results on each word; The penalty coefficient setting module is used to set the penalty coefficient of each component type; A probability adjustment module, used to calculate the probability distribution of the next word using the text generation model and adjust the probability according to the penalty coefficient corresponding to the component type of the word; The generation processing module is used to generate the next character or word using a text generation algorithm according to the adjusted probability distribution until the entire text is rewritten.

[0042] In some embodiments, the preprocessing module includes a receiving unit and a word segmentation processing unit; A receiving unit, used for receiving original text input by a user; The word segmentation processing unit is used to use the word segmentation tool to perform word segmentation on the text to obtain a word sequence.

[0043] In some embodiments, the component analysis module includes an analysis unit, a division unit, and a label setting unit; An analysis unit, for applying a syntactic analyzer to parse the text and identify the syntactic components of each word; A division unit is used to divide the identified syntactic components into component types, namely, main components and modifying components; The label setting unit is used to mark the corresponding component type on each word to form a word sequence with syntactic component type labels.

[0044] In some embodiments, the penalty coefficient setting module sets the penalty coefficient k1 of the main component and the penalty coefficient k2 of the modified component; wherein the value ranges of k1 and k2 are both between 0 and 1, and k1>k2.

[0045] In some embodiments, the probability adjustment module includes an execution unit, a tag checking unit, and an adjustment processing unit; An execution unit, for calculating the probability distribution of the next word using a text generation model; A label checking unit, used for checking the syntactic component type label of each word; The adjustment processing unit is used to, if it is a main component, multiply its probability by the penalty coefficient of the main component to obtain an adjusted probability distribution; if it is a modified component, multiply its probability by the penalty coefficient of the modified component to obtain an adjusted probability distribution; if it is unmarked, multiply its probability by 1 to obtain an adjusted probability distribution.

[0046] In some embodiments, the generation processing module includes a calculation unit, an addition processing unit, a text sorting unit, and an output unit; A computing unit, used for generating the next character or word by using a text generation algorithm according to the adjusted probability distribution; An adding processing unit, used for adding the newly generated characters or words to the generated text sequence until the entire text is rewritten; A text collating unit, used for collating the generated text; The output unit is used to output the sorted text to a specified location or format.

[0047] In some embodiments, the text collating unit is specifically used to remove redundant spaces and punctuation marks from the generated text; and to format or typeset the text as needed.

[0048] The embodiment of the present invention also provides an electronic device, which includes: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus. The communication bus can be used for information transmission between the electronic device and the sensor. The processor can call the logic instructions in the memory to execute the following method: Step 1: pre-process the input text to generate a word sequence; Step 2: perform syntactic component analysis on the text, divide the component type according to the analysis result and mark the type result on each word; Step 3: set the penalty coefficient for each component type; Step 4: use the text generation model to calculate the probability distribution of the next word and adjust the probability according to the penalty coefficient corresponding to the component type of the word; Step 5: use the text generation algorithm to generate the next character or word according to the adjusted probability distribution until the entire text is rewritten.

[0049] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program code.

[0050] An embodiment of the present invention provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions enable a computer to execute the method provided by the above method embodiment, for example, including: Step 1: preprocessing the input text to generate a word sequence; Step 2: performing syntactic component analysis on the text, dividing the component type according to the analysis results and marking the type result on each word; Step 3: setting a penalty coefficient for each component type; Step 4: using a text generation model to calculate the probability distribution of the next word and adjusting the probability according to the penalty coefficient corresponding to the component type of the word; Step 5: using a text generation algorithm to generate the next character or word according to the adjusted probability distribution, until the entire text is rewritten.

[0051] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0052] An embodiment of a device for controlling diversity of text generation provided in an embodiment of the present invention belongs to the same inventive concept as the method for controlling diversity of text generation in the above-mentioned embodiments. For details not described in detail in the embodiment of the device for controlling diversity of text generation, reference can be made to the embodiment of the method for controlling diversity of text generation in the above-mentioned embodiments.

[0053] The diversity control device for text generation is a unit and algorithm step of each example described in combination with the embodiments disclosed in this article, which can be implemented by electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0054] Those skilled in the art will appreciate that various aspects of the text generation diversity control device can be implemented as a system, method or program product. Therefore, various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which can be collectively referred to as "circuit", "module" or "system".

[0055] Although the present invention has been described in detail with reference to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention. Any person of ordinary skill in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and these shall be within the scope of protection of the present invention.

Claims

1. A diversity control method for text generation, characterized in that: The steps include: Preprocess the input text to generate word sequences; Perform syntactic component analysis on the text, divide the component types according to the analysis results and mark the type results on each word; Set the penalty coefficient for each component type; Use the text generation model to calculate the probability distribution of the next word and adjust the probability according to the penalty coefficient corresponding to the word component type; The next character or word is generated using a text generation algorithm according to the adjusted probability distribution until the entire text is rewritten.

2. The diversity control method for text generation according to claim 1, characterized in that: The steps of preprocessing the input text to generate word sequences include: Receive raw text input from the user; Use the word segmentation tool to segment the text and obtain a word sequence.

3. The diversity control method for text generation according to claim 2, characterized in that: The steps of analyzing the syntactic components of the text, dividing the component types according to the analysis results and marking the type results on each word include: Apply a syntactic analyzer to parse the text and identify the syntactic components of each word; The identified syntactic components are divided into component types, namely, main components and modifying components; The corresponding component type is marked on each word to form a word sequence with syntactic component type labels.

4. The diversity control method for text generation according to claim 3, characterized in that: The steps to set the penalty coefficient for each component type include: Set the penalty coefficient k1 for the main component and the penalty coefficient k2 for the modified component; the value ranges of k1 and k2 are both between 0 and 1, and k1 > k2.

5. The diversity control method for text generation according to claim 4, characterized in that: The steps of using the text generation model to calculate the probability distribution of the next word and adjusting the probability according to the penalty coefficient corresponding to the component type of the word include: Use the text generation model to calculate the probability distribution of the next word; For each word, check its syntactic component type label; If it is a main component, multiply its probability by the penalty coefficient of the main component to obtain the adjusted probability distribution; If it is a modification component, multiply its probability by the penalty coefficient of the modification component to obtain the adjusted probability distribution; If it is not marked, multiply its probability by 1 to get the adjusted probability distribution.

6. The diversity control method for text generation according to claim 5, characterized in that: The steps of generating the next character or word using a text generation algorithm according to the adjusted probability distribution until the entire text rewriting is completed include: Generate the next character or word using a text generation algorithm according to the adjusted probability distribution; Add the newly generated characters or words to the generated text sequence until the entire text is rewritten; Organize the generated text; Output the collated text to a specified location or format.

7. The diversity control method for text generation according to claim 6, characterized in that: The steps to organize the generated text include: Remove extra spaces and punctuation marks from the generated text; Format or format the text as desired.

8. A diversity control device for text generation, characterized in that: It includes a preprocessing module, a component analysis module, a penalty coefficient setting module, a probability adjustment module and a generation processing module; A preprocessing module is used to preprocess the input text to generate a word sequence; The component analysis module is used to perform syntactic component analysis on the text, divide the component types according to the analysis results and mark the type results on each word; The penalty coefficient setting module is used to set the penalty coefficient of each component type; A probability adjustment module, used to calculate the probability distribution of the next word using the text generation model and adjust the probability according to the penalty coefficient corresponding to the component type of the word; The generation processing module is used to generate the next character or word using a text generation algorithm according to the adjusted probability distribution until the entire text is rewritten.

9. An electronic device, characterized in that: The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor to enable the at least one processor to execute the diversity control method for text generation as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, characterized in that: The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the diversity control method for text generation according to any one of claims 1 to 7.

Images