Text generation method and apparatus, device, and storage medium
By traversing the method of combining and punishing duplicate words, the problem of high similarity in text generated by cluster search method is solved, and the diversity and readability of the text is improved.
Patent Information
- Application Number
- PCT/CN2024/092636
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-12
- Filing Date
- 2024-05-11
- Publication Date
- 2024-11-21
AI Technical Summary
The existing cluster search methods generate high similarity in texts, resulting in low text diversity. The generated text often has only one word different or the semantics are exactly the same.
By obtaining the text generated at the previous moment, determining its candidate words, and traversing and combining them, punishing the repeated words, and calculating the set score of the candidate text set, thereby generating diverse text.
Improves the diversity of text generation, ensures that the generated text contains no duplicate words, and enhances the readability and fluency of the text.
Smart Images

Figure CN2024092636_21112024_PF_FP_ABST
Abstract
Description
Text generation method, device, equipment and storage medium
[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on May 12, 2023, with application number 202310538686.9, the entire contents of which are incorporated by reference into this application. Technical Field
[0002] Embodiments of the present application relate to computer technology, for example, to a text generation method, apparatus, device, and storage medium. Background Art
[0003] With the rapid development of computer technology, it is often necessary to automatically generate some text to meet different business needs. For example, the generated text can be used for machine learning or text translation.
[0004] Currently, text is usually generated through beam search, which is used to search for a sequence of generated text sequences with the highest conditional probability value.
[0005] However, in the process of implementing this application, the inventors discovered that there are at least the following problems in the related art:
[0006] The text similarity generated by the beam search method in the related art is very high. For example, there is only one word different in two generated long text sentences or the text semantics are exactly the same, which leads to low diversity of the generated text.
[0007] Summary of the Invention
[0008] Embodiments of the present application provide a text generation method, apparatus, device, and storage medium to improve the diversity of generated text.
[0009] In a first aspect, an embodiment of the present application provides a text generation method, comprising:
[0010] Get the preset number of first texts generated at the last moment;
[0011] Determining a plurality of first candidate words corresponding to each of the first texts, and using each of the first candidate words as a suffix of the first text to generate a plurality of first candidate texts;
[0012] Traversing and combining a plurality of first candidate texts corresponding to the preset number of first texts to determine a plurality of candidate text sets, wherein each candidate text set includes the preset number of first candidate texts;
[0013] Based on each word in each first candidate text, determining repeated words in each candidate text set, and penalizing word scores corresponding to the repeated words to determine a target score corresponding to the repeated words, wherein the word scores corresponding to the repeated words are used to represent the conditional probability value of the repeated words as text suffix words;
[0014] Determine a set score corresponding to each candidate text set based on the target scores corresponding to the repeated words in each candidate text set and the word scores corresponding to the non-repeated words;
[0015] Based on the set scores, a target text set is determined from a plurality of candidate text sets, and based on the target text set, the preset number of second texts generated at the current moment is determined.
[0016] In a second aspect, an embodiment of the present application further provides a text generation device, comprising:
[0017] A first text acquisition module, configured to acquire a preset number of first texts generated at a previous moment;
[0018] A first candidate text generating module is configured to determine a plurality of first candidate words corresponding to each of the first texts, and use each of the first candidate words as a suffix of the first text to generate a plurality of first candidate texts;
[0019] a candidate text set determining module configured to traverse and combine a plurality of first candidate texts corresponding to the preset number of first texts to determine a plurality of candidate text sets, wherein each candidate text set includes the preset number of first candidate texts;
[0020] a target score determination module configured to determine repeated words in each candidate text set based on each word in each first candidate text, and to perform a penalty process on word scores corresponding to the repeated words to determine a target score corresponding to the repeated words, wherein the word scores corresponding to the repeated words are used to represent a conditional probability value of the repeated words as text suffix words;
[0021] A set score determination module configured to determine a set score corresponding to each candidate text set based on target scores corresponding to repeated words and word scores corresponding to non-repeated words in each candidate text set;
[0022] The second text determination module is configured to determine a target text set from multiple candidate text sets based on the set scores, and determine the preset number of second texts generated at the current moment based on the target text set.
[0023] In a third aspect, an embodiment of the present application further provides an electronic device, comprising:
[0024] at least one processor;
[0025] a memory configured to store at least one program;
[0026] When the at least one program is executed by the at least one processor, the at least one processor implements the text generation method provided in any embodiment of the present application.
[0027] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored, which, when executed by a processor, implements a text generation method as provided in any embodiment of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0028] FIG1 is a flow chart of a text generation method provided by one embodiment of the present application;
[0029] FIG2 is an example of text generation according to an embodiment of the present application;
[0030] FIG3 is a flowchart of another text generation method provided by an embodiment of the present application;
[0031] FIG4 is an example of multiple second candidate words corresponding to a second text involved in an embodiment of the present application;
[0032] FIG5 is an example of a target second candidate word corresponding to a second text involved in an embodiment of the present application;
[0033] FIG6 is a schematic structural diagram of a text generation device provided by an embodiment of the present application;
[0034] FIG7 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION
[0035] The present application will be described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to explain the present application and are not intended to limit the present application. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions of the present application, not all of the structures.
[0036] FIG1 is a flowchart of a text generation method provided by one embodiment of the present application. This embodiment is applicable to generating texts with diverse content. The method can be performed by a text generation device, which can be implemented by software and / or hardware and integrated into an electronic device. As shown in FIG1 , the method includes the following steps:
[0037] S110: Obtain a preset number of first texts generated at the last moment.
[0038] Among them, the preset number can be a pre-set number of texts generated at each moment. For example, the preset number can also be used to characterize the bundle width. The previous moment can refer to the moment when the text was generated last time. The current moment can refer to the moment when the text is generated this time. The previous moment can refer to the previous moment adjacent to the current moment. For example, the current moment is moment t, and the previous moment is moment t-1. The first text can refer to the text generated at the previous moment. The text in this embodiment can be represented by a word sequence consisting of at least one word. The complete text needs to be repeatedly generated on the basis of the first text generated at the previous moment. For example, when generated for the first time, the first text corresponding to the previous moment can be an empty text, so that a new text can be generated from scratch; or it can be a preset existing text, so that text can be generated based on the existing text. As time goes by, the length of the generated text becomes longer and longer.
[0039] S120: Determine a plurality of first candidate words corresponding to each first text, and use each first candidate word as a suffix of the first text to generate a plurality of first candidate texts.
[0040] The first candidate word refers to a word that can be used as a suffix of the first text. The first candidate text can be the text generated at the current moment.
[0041] Exemplarily, all first candidate words corresponding to each first text can be determined based on the dictionary, and each first candidate word can be used as a suffix of the corresponding first text for splicing and combination to obtain all first candidate texts corresponding to each first text. For example, Figure 2 gives an example of text generation. As shown in Figure 2, there are two first texts, namely: "this model" and "wireless", and the first candidate words corresponding to "this model" are: "headphones", "headsets", etc. The first candidate words corresponding to "wireless" are: "headphones", "Bluetooth", etc. Taking each first candidate word as a suffix of the corresponding first text, the first candidate texts obtained are: "this model", "this model", "wireless headphones", "wireless Bluetooth", etc.
[0042] S130 , traverse and combine multiple first candidate texts corresponding to a preset number of first texts to determine multiple candidate text sets, wherein each candidate text set includes a preset number of first candidate texts.
[0043] Exemplarily, all first candidate texts corresponding to all first texts can be traversed and combined exhaustively to obtain a set of all candidate texts containing a preset number of first candidate texts. For example, if the preset number is 2, all first candidate texts corresponding to all first texts are combined in pairs to obtain a set of candidate texts corresponding to each combination, so as to generate text more comprehensively. It is also possible to traverse and select a first candidate text from all first candidate texts corresponding to each first text for combination to obtain a set of candidate texts corresponding to each selection situation, so that each first candidate text in the candidate text set has a different text prefix, thereby ensuring the diversity of text generation. It should be noted that there can be at least one different first candidate text in the two candidate text sets. For example, Figure 2 only shows two candidate text sets, namely: {"this headset", "wireless headset"} and {"this headset", "wireless Bluetooth"}.
[0044] S140. Determine repeated words in each candidate text set, and penalize the word scores corresponding to the repeated words to determine the target scores corresponding to the repeated words, wherein the word scores corresponding to the repeated words are used to represent the conditional probability value of the repeated words as text suffix words.
[0045] Among them, repeated words can refer to words that appear repeatedly in the candidate text set. The word score corresponding to the repeated word can be used to represent the conditional probability value of the repeated word as a text suffix word. For example, the conditional probability value can be directly used as the word score, or the score obtained by taking the logarithm of the conditional probability value can be used as the word score. The word score can be used to reflect the readability and fluency of the text. For example, the higher the word score, the higher the readability and fluency of the text generated using the word. The target score corresponding to the repeated word can refer to the score obtained after the word score is penalized. The target score corresponding to the repeated word is less than the corresponding word score.
[0046] Exemplarily, for each candidate text set, all words in each first candidate text in the candidate text set can be compared to determine all repeated words that appear repeatedly in the candidate text set, and the word score corresponding to each repeated word is penalized to determine the target score corresponding to each repeated word. For example, any score that is less than the word score of the repeated word can be used as the target score corresponding to the repeated word. For example, the score obtained by subtracting a preset score from the word score corresponding to each repeated word can be used as the target score corresponding to the repeated word.
[0047] It should be noted that the word score corresponding to each word can be obtained by taking the logarithm of the conditional probability value, and the word score obtained in this way is a negative number. The larger the negative number, the higher the conditional probability value, and the higher the readability and fluency. For example, the word score corresponding to "headphones" in "this type of headphones" in Figure 2 is: -1.8, which represents the score of the word "headphones" as the suffix of "this type". The word score corresponding to "headphones" in "wireless headphones" is: -1.9, which represents the score of the word "headphones" as the suffix of "wireless".
[0048] Exemplarily, "performing penalty processing on word scores corresponding to repeated words and determining target scores corresponding to repeated words" in S140 may include: obtaining the word scores corresponding to each occurrence position of the current repeated word; multiplying the word scores corresponding to at least one preset occurrence position by a preset penalty coefficient to obtain the target score corresponding to the current repeated word at the preset occurrence position; and using the word scores corresponding to the remaining occurrence positions other than the preset occurrence positions as the target scores corresponding to the current repeated word at the remaining occurrence positions.
[0049] Among them, each repeated word can be treated as the current repeated word and subjected to word score penalty processing. The occurrence position can refer to the occurrence position of the repeated word in the candidate text set. Different occurrence positions may correspond to different word scores. For example, for the candidate text set {"this headset", "wireless headset"} in Figure 2, the repeated word is "headphone", and the repeated word appears twice. The first occurrence position is in "this headset", and the corresponding word score is: -1.8; the second occurrence position is in: "wireless headset", and the corresponding word score is: -1.9. The preset occurrence position can be a pre-set occurrence position that needs to be penalized for word scores. The number of preset occurrence positions can be at least one, which is set based on business needs. For example, the preset occurrence position is the first occurrence position or the last occurrence position, etc. The remaining occurrence positions refer to all occurrence positions except the preset occurrence position. The preset penalty coefficient can be a pre-set coefficient greater than 1. For example, the preset penalty coefficient is 10.
[0050] For example, as shown in Figure 2, when the preset appearance position is the last appearance position, the target score corresponding to the repeated word "headphones" at the first appearance position is determined to be: the original word score, that is, -1.8. The target score corresponding to the second appearance position, that is, the last appearance position, is: the product of the original word score -1.9 and the preset penalty coefficient 10, that is, -19. By only penalizing the word scores at the preset appearance positions, the efficiency of the penalty processing can be improved while maintaining diversity.
[0051] S150 : Determine a set score corresponding to each candidate text set based on the target scores corresponding to the repeated words and the word scores corresponding to the non-repeated words in each candidate text set.
[0052] For example, the target scores corresponding to all repeated words and the word scores corresponding to all non-repeated words in each candidate text set can be integrated to obtain a set score corresponding to each candidate text set. In this embodiment, the word scores of repeated words in the candidate text set can be penalized only, without penalizing the word scores of non-repeated words. This can tend to generate texts containing non-repeated words, thereby improving the diversity of the generated texts.
[0053] Exemplarily, S150 may include: adding target scores corresponding to all repeated words and word scores corresponding to all non-repeated words in the current candidate text set, and determining the obtained addition result as the set score corresponding to the current candidate text set.
[0054] For example, each candidate text set can be used as the current candidate text set to calculate the set score. For example, the set score corresponding to the candidate text set {"this headset", "wireless headset"} in FIG2 is: -3.1-3.2-1.8-19=-27.1.
[0055] If all the words in the current candidate text set are distinct and there are no duplicates, the word scores for all the distinct words in the current candidate text set can be summed up. The summed result is used as the set score for the current candidate text set. The set score for the candidate text set {"this headset," "wireless Bluetooth"} in Figure 2 is: -3.1 - 3.2 - 1.8 - 2.2 = -10.3.
[0056] S160 : Determine a target text set from multiple candidate text sets based on the set scores, and determine a preset number of second texts generated at the current moment based on the target text set.
[0057] The target text set may refer to the final text set generated at the current moment, and the second text may refer to the final text generated at the current moment.
[0058] For example, based on the set score corresponding to each candidate text set, an optimal target text set is selected, and a preset number of second texts generated at the current moment are obtained based on the target text set. By penalizing repeated words with word scores and selecting based on the set score, second texts containing non-repeated words are favored, significantly increasing the diversity of the generated second texts.
[0059] Exemplarily, S160 may include: determining the candidate text set with the highest set score as the target text set, and determining the target texts in the target text set as a preset number of second texts generated at the current moment. All candidate text sets can be sorted based on the set score, and the candidate text set with the highest set score is used as the final target text set, and each target text in the target text set is used as the second text generated at the current moment, thereby generating diversified second texts. For example, the candidate text set {"this headset", "wireless Bluetooth"} in Figure 2 is determined as the target text set, and the two target texts in the set are the two second texts generated at the current moment.
[0060] It should be noted that the text at the next moment can be continuously generated based on the second text generated at the current moment by looping through steps S110-S160 until the number of generations equals the preset number or the length of the generated text equals the preset complete text length.
[0061] The technical solution of this embodiment generates multiple first candidate texts based on a preset number of first texts generated at the previous moment within the current time step, and traverses and combines the multiple first candidate texts corresponding to the preset number of first texts to obtain multiple candidate text sets, performs word score penalty processing on repeated words appearing in each candidate text set, and determines the set score corresponding to each candidate text set based on the target score after penalty, determines the target text set from the multiple candidate text sets based on the set score, and determines the preset number of second texts generated at the current moment based on the target text set, so that by performing word score penalty processing on repeated words appearing in each candidate text set, it can tend to generate texts containing non-repeated words, thereby improving the diversity of generated texts.
[0062] Based on the above technical solution, after S160, the following steps may be further included:
[0063] S170: If the length of the second text is greater than or equal to the preset length, determine a plurality of second candidate words corresponding to each second text.
[0064] The preset length may be a minimum length that is pre-set and stops the loop generation of text using the above steps S110-S160. The second candidate word refers to a word that can be used as a suffix of the second text.
[0065] It should be noted that when the generated text reaches a certain length, the above-mentioned generation method of improving diversity will affect the readability of the generated text to a certain extent. Therefore, after exceeding a certain length, it is necessary to continue to generate text using a generation method that ensures readability, so as to ensure the readability of the text while improving diversity.
[0066] Exemplarily, after generating the second text at the current moment, it is possible to detect whether the length of the second text generated at the current moment is greater than or equal to a preset length. If the length of the second text generated at the current moment is greater than or equal to the preset length, the text of the next moment is generated using the method of steps S170-S190. If the length of the second text generated at the current moment is less than the preset length, the text of the next moment is generated using the method of steps S110-S160 described above. If the length of the second text is greater than or equal to the preset length, multiple second candidate words corresponding to each second text can be determined based on the dictionary.
[0067] S180: Use each second candidate word as a suffix of the second text to generate multiple second candidate texts, and determine a text score corresponding to each second candidate text based on the word score corresponding to each word in each second candidate text.
[0068] For example, each second candidate word can be concatenated as a suffix of the corresponding second text to obtain all second candidate texts corresponding to each second text. For each second candidate text, the word scores corresponding to each word in the second candidate text can be added together, and the sum obtained is used as the text score corresponding to the second candidate text. The text score can be used to represent the readability of the second candidate text. For example, the higher the text score, the more readable the second candidate text.
[0069] S190 : Determine, based on the text scores, a preset number of third texts to be generated at a next moment from a plurality of second candidate texts corresponding to the preset number of second texts.
[0070] For example, all second candidate texts can be sorted in descending order based on their text scores to obtain a sequence of second candidate texts with successively lower text scores, and the first predetermined number of second candidate texts in the sequence of second candidate texts are determined as the third texts to be generated at the next moment. By selecting second candidate texts with higher text scores as the third texts, text readability can be ensured while increasing diversity.
[0071] It should be noted that after the next moment, the text of subsequent moments can be continuously generated based on the third text generated at the next moment by looping steps S170-S190 until the number of generations is equal to the preset number or the length of the generated text is equal to the preset complete text length.
[0072] FIG3 is a flowchart of another text generation method provided by an embodiment of the present application. Based on the above embodiments, this embodiment further refines the step of "using each second candidate word as a suffix of the second text to generate multiple second candidate texts, and determining the text score corresponding to each second candidate text based on the word score corresponding to each word in each second candidate text." Furthermore, based on this, the step of "determining a preset number of third texts to be generated at the next moment from the multiple second candidate texts based on the text score" is also refined. Explanations of terms that are identical or corresponding to those in the above embodiments are not repeated here.
[0073] Referring to FIG3 , another text generation method provided in this embodiment includes the following steps:
[0074] S310: Obtain a preset number of first texts generated at the last moment.
[0075] S320: Determine a plurality of first candidate words corresponding to each first text, and use each first candidate word as a suffix of the first text to generate a plurality of first candidate texts.
[0076] S330: Traverse and combine multiple first candidate texts corresponding to a preset number of first texts to determine multiple candidate text sets, wherein each candidate text set includes a preset number of first candidate texts.
[0077] S340: Determine repeated words in each candidate text set, and perform penalty processing on word scores corresponding to the repeated words to determine target scores corresponding to the repeated words.
[0078] S350: Determine a set score corresponding to each candidate text set based on the target scores corresponding to the repeated words and the word scores corresponding to the non-repeated words in each candidate text set.
[0079] S360: Determine a target text set from the plurality of candidate text sets based on the set scores, and determine a preset number of second texts generated at the current moment based on the target text set.
[0080] S370: If the length of the second text is greater than or equal to the preset length, determine a plurality of second candidate words corresponding to each second text.
[0081] For example, FIG4 illustrates a plurality of second candidate words corresponding to a second text. The preset number in FIG4 is 2, indicating that there are two second texts. In FIG4 , second text 1 includes two second texts, and second text 2 also includes two second texts. Each second text independently determines a corresponding second candidate word and independently performs subsequent steps S380-S391.
[0082] S380: Use each current second candidate word corresponding to the current second text as a suffix of the current second text to generate a current second candidate text corresponding to the current second text.
[0083] It should be noted that each second text may be used as the current second text, and steps S380 - S391 may be executed in parallel to determine the third candidate text corresponding to each second text.
[0084] Exemplarily, each current second candidate word corresponding to the current second text may be concatenated and combined as a suffix of the current second text to obtain all current second candidate texts corresponding to the current second text.
[0085] S390: Determine a text score corresponding to each current second candidate text based on the word score corresponding to each word in each current second candidate text.
[0086] For example, for each current second candidate text, the word scores corresponding to each word in the current second candidate text can be added together, and the sum obtained is used as the text score corresponding to the current second candidate text. The text score can be used to represent the readability of the current second candidate text. For example, the higher the text score, the more readable the current second candidate text.
[0087] S391: Determine a preset number of current third candidate texts corresponding to the current second text from a plurality of current second candidate texts based on the text scores.
[0088] For example, all current second candidate texts can be sorted in descending order based on their text scores to obtain a current second candidate text sequence with successively decreasing text scores, and the first preset number of current second candidate texts in the current second candidate text sequence are determined as the current third candidate texts corresponding to the current second text, thereby independently generating a corresponding preset number of third candidate texts for each second text. The second texts do not affect each other, thereby increasing the diversity of the third candidate texts between different second texts. Furthermore, even after the text length exceeds the preset length, the diversity of the text can be further ensured while ensuring the readability of the text, thereby avoiding a decrease in the diversity of the generated text.
[0089] S392: Determine a preset number of third texts to be generated at the next moment from the third candidate texts corresponding to the preset number of second texts.
[0090] Exemplarily, an optimal preset number of third texts may be selected from all third candidate texts based on the text score corresponding to each third candidate text corresponding to each second text.
[0091] For example, S392 may include: arranging all third candidate texts corresponding to a preset number of second texts in descending order based on the text scores corresponding to each third candidate text corresponding to each second text to obtain a third candidate text sequence; and determining the first preset number of third candidate texts in the third candidate text sequence as the third text to be generated at the next moment. By determining the first preset number of third candidate texts with the highest text scores as the third text to be generated at the next moment, text diversity can be further ensured while ensuring text readability.
[0092] The technical solution of this embodiment generates a corresponding preset number of third candidate texts for each second text independently, and determines the preset number of third texts to be generated at the next moment from all the third candidate texts. Therefore, even after the text length exceeds the preset length, the diversity of the text can be further guaranteed while ensuring the readability of the text.
[0093] Based on the above technical solution, S380 may include: obtaining a preset number of used words in the second text; comparing each current second candidate word corresponding to the current second text with the used words to determine a target second candidate word that is different from the used words; using each target second candidate word as a suffix of the current second text to generate a current second candidate text corresponding to the current second text.
[0094] The used words may refer to words that have been used in the second text, and the target second candidate words may refer to the current second candidate words remaining after removing the used words from all the current second candidate words.
[0095] Figure 5 shows an example of target second candidate words corresponding to a second text. As shown in Figure 5, all words contained in second text 1 and second text 2 are already used words. For second text 1, none of the second candidate words corresponding to second text 1 are already used words. Therefore, each second candidate word corresponding to second text 1 can be directly used as the corresponding target second candidate word. For second text 2, the second candidate word "Lightweight" is already used. In this case, "Lightweight" can be deleted, and the remaining second candidate words can be used as the target second candidate words corresponding to second text 2. After determining the target second candidate words, each target second candidate word corresponding to the current second text can be used as a suffix of the current second text for concatenation and combination to obtain all current second candidate words corresponding to the current second text. By removing already used words from the second candidate words corresponding to the second text, deduplication is achieved, avoiding the recurrence of the same word, increasing the diversity of the current second candidate texts generated from different second texts, and thus increasing the diversity of the generated third texts. This allows for further improvement in text diversity while ensuring readability even when the text length exceeds a preset length.
[0096] The following is an embodiment of the text generation device provided in the embodiments of the present application. The device and the text generation methods of the above embodiments belong to the same inventive concept. For details not fully described in the embodiments of the text generation device, please refer to the embodiments of the above text generation method.
[0097] FIG6 is a schematic diagram of the structure of a text generation device provided in an embodiment of the present application. This embodiment is applicable to generating texts with diverse content. As shown in FIG6 , the device includes: a first text acquisition module 610, a first candidate text generation module 620, a candidate text set determination module 630, a target score determination module 640, a set score determination module 650, and a second text determination module 660.
[0098] Among them, the first text acquisition module 610 is configured to acquire a preset number of first texts generated at the previous moment; the first candidate text generation module 620 is configured to determine multiple first candidate words corresponding to each of the first texts, and use each of the first candidate words as a suffix of the first text to generate multiple first candidate texts; the candidate text set determination module 630 is configured to traverse and combine the multiple first candidate texts corresponding to the preset number of first texts to determine multiple candidate text sets, wherein each candidate text set includes the preset number of first candidate texts; the target score determination module 640 is configured to determine the score of each candidate text based on each word in each first candidate text. Repeated words that appear repeatedly in a text set are selected, and word scores corresponding to the repeated words are penalized to determine target scores corresponding to the repeated words, wherein the word scores corresponding to the repeated words are used to characterize the conditional probability value of the repeated words as text suffix words; a set score determination module 650 is configured to determine the set score corresponding to each candidate text set based on the target scores corresponding to the repeated words and the word scores corresponding to the non-repeated words in each candidate text set; a second text determination module 660 is configured to determine a target text set from multiple candidate text sets based on the set scores, and determine a preset number of second texts generated at the current moment based on the target text set.
[0099] The technical solution of this embodiment generates multiple first candidate texts based on a preset number of first texts generated at the previous moment within the current time step, and traverses and combines the multiple first candidate texts corresponding to the preset number of first texts to obtain multiple candidate text sets, performs word score penalty processing on repeated words appearing in each candidate text set, and determines the set score corresponding to each candidate text set based on the target score after penalty, determines the target text set from the multiple candidate text sets based on the set score, and determines the preset number of second texts generated at the current moment based on the target text set, so that by performing word score penalty processing on repeated words appearing in each candidate text set, it is possible to tend to generate texts containing non-repeated words, thereby improving the diversity of generated texts.
[0100] Optionally, the target score determination module 640 is configured to:
[0101] Obtain the word score corresponding to each occurrence position of the current repeated word; multiply the word score corresponding to at least one preset occurrence position by a preset penalty coefficient to obtain the target score corresponding to the current repeated word at the preset occurrence position; and use the word scores corresponding to the remaining occurrence positions other than the preset occurrence positions as the target scores corresponding to the current repeated word at the remaining occurrence positions.
[0102] Optionally, the set score determination module 650 is configured to:
[0103] The target scores corresponding to the repeated words and the word scores corresponding to the non-repeated words in the current candidate text set are added together, and the obtained addition result is determined as the set score corresponding to the current candidate text set.
[0104] Optionally, the second text determination module 660 is configured to:
[0105] The candidate text set with the highest set score is determined as the target text set, and each target text in the target text set is determined as a preset number of second texts generated at the current moment.
[0106] Optionally, the device further comprises:
[0107] A second candidate word determination module is configured to, after determining a preset number of second texts generated at a current moment, determine a plurality of second candidate words corresponding to each second text if the length of the second text is greater than or equal to a preset length;
[0108] a text score determination module configured to use each of the second candidate words as a suffix of the second text to generate a plurality of second candidate texts, and determine a text score corresponding to each second candidate text based on a word score corresponding to each word in each second candidate text;
[0109] The third text determination module is configured to determine a preset number of third texts to be generated at a next moment from a plurality of second candidate texts based on the text scores.
[0110] Optionally, the text score determination module includes:
[0111] a current second candidate text determining unit, configured to use each current second candidate word corresponding to the current second text as a suffix of the current second text to generate a current second candidate text corresponding to the current second text;
[0112] a text score determining unit configured to determine a text score corresponding to each current second candidate text based on a word score corresponding to each word in each current second candidate text;
[0113] The third text determination module includes:
[0114] a current third candidate text determining unit, configured to determine a preset number of current third candidate texts corresponding to the current second text from a plurality of current second candidate texts based on the text scores;
[0115] The third text determining unit is configured to determine a preset number of third texts to be generated at a next moment from the third candidate texts corresponding to each second text.
[0116] Optionally, the current second candidate text determination unit is set to:
[0117] Obtain the used words in each second text; compare each current second candidate word corresponding to the current second text with the used words to determine a target second candidate word that is different from the used words; use each target second candidate word as a suffix of the current second text to generate a current second candidate text corresponding to the current second text.
[0118] Optionally, the third text determination unit is configured to:
[0119] Based on the text scores corresponding to each third candidate text corresponding to each second text, the third candidate texts are arranged in descending order to obtain a third candidate text sequence; and the first preset number of third candidate texts in the third candidate text sequence are determined as the third texts to be generated at the next moment.
[0120] The text generation device provided in the embodiments of the present application can execute the text generation method provided in any embodiment of the present application, and has corresponding functional modules for executing the text generation method.
[0121] It is worth noting that in the embodiment of the above-mentioned text generation device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application.
[0122] FIG7 is a block diagram of an electronic device according to an embodiment of the present application. FIG7 shows a block diagram of an exemplary electronic device 12 suitable for implementing the embodiments of the present application. The electronic device 12 shown in FIG7 is merely an example and should not limit the functionality or scope of use of the embodiments of the present application.
[0123] 7 , electronic device 12 is implemented as a general-purpose computing device. Components of electronic device 12 may include, but are not limited to, at least one processor or processing unit 16, system memory 28, and a bus 18 connecting various system components (including system memory 28 and processing unit 16).
[0124] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor, or a local bus using any of a variety of bus architectures. Examples of these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.
[0125] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.
[0126] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write to non-removable, non-volatile magnetic media (not shown in FIG. 7 , commonly referred to as a “hard drive”). Although not shown in FIG. 7 , a magnetic disk drive for reading and writing to a removable non-volatile magnetic disk (e.g., a “floppy disk”) may be provided, as well as an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a compact disc read-only memory (CD-ROM), a digital video disc (DVD-ROM), or other optical media). In these cases, each drive may be connected to the bus 18 via one or more data media interfaces. The system memory 28 may include at least one program product having a set (eg, at least one) of program modules. These program modules are configured to perform the functions of various embodiments of the present application.
[0127] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may include an implementation of a network environment. Program modules 42 generally implement the functions and / or methods of the embodiments described herein.
[0128] The electronic device 12 can also communicate with one or more external devices 14 (e.g., a keyboard, pointing device, display 24, etc.), one or more devices that enable a user to interact with the electronic device 12, and / or any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). This communication can occur via an input / output (I / O) interface 22. Furthermore, the electronic device 12 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 via a bus 18. It should be understood that, although not shown, other hardware and / or software modules can be used in conjunction with the electronic device 12, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, Redundant Arrays of Independent Disks (RAID) systems, tape drives, and data backup storage systems.
[0129] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing a text generation method provided in an embodiment of the present invention, which includes:
[0130] Get the preset number of first texts generated at the last moment;
[0131] Determining a plurality of first candidate words corresponding to each of the first texts, and using each of the first candidate words as a suffix of the first text to generate a plurality of first candidate texts;
[0132] Traversing and combining a plurality of first candidate texts corresponding to the preset number of first texts to determine a plurality of candidate text sets, wherein each candidate text set includes the preset number of first candidate texts;
[0133] Based on each word in each first candidate text, determining repeated words in each candidate text set, and penalizing word scores corresponding to the repeated words to determine a target score corresponding to the repeated words, wherein the word scores corresponding to the repeated words are used to represent the conditional probability value of the repeated words as text suffix words;
[0134] Determine a set score corresponding to each candidate text set based on the target scores corresponding to the repeated words in each candidate text set and the word scores corresponding to the non-repeated words;
[0135] Based on the set scores, a target text set is determined from a plurality of candidate text sets, and based on the target text set, a preset number of second texts generated at a current moment is determined.
[0136] Of course, those skilled in the art will understand that the processor can also implement the technical solution of the text generation method provided in any embodiment of the present application.
[0137] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the text generation method provided in any embodiment of the present application are implemented. The method includes:
[0138] Get the preset number of first texts generated at the last moment;
[0139] Determining a plurality of first candidate words corresponding to each of the first texts, and using each of the first candidate words as a suffix of the first text to generate a plurality of first candidate texts;
[0140] Traversing and combining a plurality of first candidate texts corresponding to the preset number of first texts to determine a plurality of candidate text sets, wherein each candidate text set includes the preset number of first candidate texts;
[0141] Based on each word in each first candidate text, determining repeated words in each candidate text set, and penalizing word scores corresponding to the repeated words to determine a target score corresponding to the repeated words, wherein the word scores corresponding to the repeated words are used to represent the conditional probability value of the repeated words as text suffix words;
[0142] Determine a set score corresponding to each candidate text set based on the target scores corresponding to the repeated words in each candidate text set and the word scores corresponding to the non-repeated words;
[0143] Based on the set scores, a target text set is determined from a plurality of candidate text sets, and based on the target text set, a preset number of second texts generated at a current moment is determined.
[0144] The computer storage medium of the embodiment of the present application can adopt any combination of one or more computer-readable media.Computer-readable media can be computer-readable signal media or computer-readable storage media.Computer-readable storage media can be, for example, but not limited to: electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above.More specific examples (non-exhaustive list) of computer-readable storage media include: electrical connections with one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM (Erasable Programmable Read-Only Memory) or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above.In this document, computer-readable storage media can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.
[0145] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device.
[0146] The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination thereof.
[0147] The computer program code for performing the operations of the present application can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional procedural programming languages such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0148] Those skilled in the art will appreciate that the modules or steps of the present application described above can be implemented using a general-purpose computing device. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Alternatively, they can be implemented using program code executable by a computer device, so that they can be stored in a storage device and executed by the computing device. Alternatively, they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.
[0149] Note that the above are only optional embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application has been described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.
Claims
1. A text generation method, comprising: Get the preset number of first texts generated at the last moment; Determine a plurality of first candidate words corresponding to each of the first texts, and use each of the first candidate words as a suffix of the first text to generate a plurality of first candidate texts; Traversing and combining a plurality of first candidate texts corresponding to the preset number of the first texts to determine a plurality of candidate text sets, wherein each candidate text set includes the preset number of first candidate texts; Determine the repeated words in each candidate text set, and perform penalty processing on the word scores corresponding to the repeated words to determine the target scores corresponding to the repeated words, wherein the word scores corresponding to the repeated words are used to represent the conditional probability value of the repeated words as text suffix words; Determine a set score corresponding to each candidate text set based on the target scores corresponding to the repeated words in each candidate text set and the word scores corresponding to the non-repeated words; Based on the set score, a target text set is determined from a plurality of candidate text sets, and based on the target text set, the preset number of second texts generated at the current moment is determined.
2. The method according to claim 1, wherein: The penalizing the word scores corresponding to the repeated words to determine the target scores corresponding to the repeated words includes: Get the word score corresponding to each occurrence position of the current repeated word; Multiplying the word score corresponding to at least one preset occurrence position by a preset penalty coefficient to obtain a target score corresponding to the at least one preset occurrence position of the current repeated word; The word scores corresponding to the remaining appearance positions except the at least one preset appearance position are used as the target scores corresponding to the current repeated word in the remaining appearance positions.
3. The method according to claim 1, wherein: The step of determining the set score corresponding to each candidate text set based on the target score corresponding to the repeated words and the word score corresponding to the non-repeated words in each candidate text set comprises: The target scores corresponding to all repeated words in the current candidate text set and the word scores corresponding to all non-repeated words are added together, and the obtained addition result is determined as the set score corresponding to the current candidate text set.
4. The method according to claim 1, wherein: The step of determining a target text set from a plurality of candidate text sets based on the set score, and determining the preset number of second texts generated at the current moment based on the target text set, comprises: The candidate text set with the highest set score is determined as a target text set, and the target texts in the target text set are determined as the preset number of second texts generated at the current moment.
5. The method according to any one of claims 1 to 4, after determining the preset number of second texts generated at the current moment, further comprising: If the length of the second text is greater than or equal to a preset length, determining a plurality of second candidate words corresponding to each of the second texts; Using each of the second candidate words as a suffix of the second text to generate a plurality of second candidate texts, and determining a text score corresponding to each of the second candidate texts based on the word score corresponding to each word in each of the second candidate texts; Based on the text scores, the preset number of third texts to be generated at the next moment are determined from a plurality of second candidate texts corresponding to the preset number of second texts.
6. The method according to claim 5, wherein: The step of using each of the second candidate words as a suffix of the second text to generate a plurality of second candidate texts, and determining a text score corresponding to each of the second candidate texts based on a word score corresponding to each word in each of the second candidate texts, includes: Using each current second candidate word corresponding to the current second text as a suffix of the current second text to generate a current second candidate text corresponding to the current second text; Determine a text score corresponding to each current second candidate text based on the word score corresponding to each word in each current second candidate text; The step of determining the preset number of third texts to be generated at the next moment from a plurality of second candidate texts corresponding to the preset number of second texts based on the text scores includes: Based on the text score, determining the preset number of current third candidate texts corresponding to the current second text from the plurality of current second candidate texts; Determine the third candidate texts generated at the next moment from the preset number of third candidate texts corresponding to the second texts The preset number of the third texts.
7. The method according to claim 6, wherein: The step of using each current second candidate word corresponding to the current second text as a suffix of the current second text to generate a current second candidate text corresponding to the current second text includes: Acquire the preset number of used words in the second text; Compare each current second candidate word corresponding to the current second text with the used word to determine a target second candidate word different from the used word; Each of the target second candidate words is used as a suffix of the current second text to generate a current second candidate text corresponding to the current second text.
8. The method according to claim 6, wherein: The step of determining the preset number of third texts to be generated at a next moment from the third candidate texts corresponding to the preset number of second texts includes: Based on the text score corresponding to each third candidate text corresponding to each second text, the third candidate texts corresponding to the preset number of the second texts are arranged in descending order to obtain a third candidate text sequence; The first preset number of third candidate texts in the third candidate text sequence are determined as the third texts to be generated at the next moment.
9. A text generation device, comprising: A first text acquisition module, configured to acquire a preset number of first texts generated at a previous moment; A first candidate text generating module, configured to determine a plurality of first candidate words corresponding to each of the first texts, and use each of the first candidate words as a suffix of the first text to generate a plurality of first candidate texts; a candidate text set determination module, configured to traverse and combine a plurality of first candidate texts corresponding to the preset number of the first texts to determine a plurality of candidate text sets, wherein each candidate text set includes the preset number of first candidate texts; The target score determination module is configured to determine the repeated words in each candidate text set based on each word in each first candidate text, and to penalize the word scores corresponding to the repeated words. The target score corresponding to the repeated word is determined, wherein the word score corresponding to the repeated word is used to represent the conditional probability value of the repeated word as a text suffix word; A set score determination module, configured to determine a set score corresponding to each candidate text set based on a target score corresponding to a repeated word in each candidate text set and a word score corresponding to a non-repeated word; The second text determination module is configured to determine a target text set from a plurality of candidate text sets based on the set score, and determine the preset number of second texts generated at the current moment based on the target text set.
10. An electronic device, comprising: at least one processor; a memory configured to store at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the text generation method as described in any one of claims 1-8.
11. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the text generation method according to any one of claims 1 to 8.
Citation Information
Patent Citations
Method and device used for generating text on the basis of artificial intelligence
CN107526725A
Cboth generation method and device, electronic equipment and storage medium
CN115238673A
Text generation method and device, equipment and storage medium
CN116562239A
Text generation method and apparatus, and electronic device and computer-readable medium
WO2022151915A1
