Text generation method and device, equipment and storage medium

Abstract

The embodiments of the present specification provide a text generation method, device and equipment and storage medium. The method comprises: obtaining a word order sequence set comprising a plurality of word order sequences generated according to a reference text and a prediction value of the word order sequence; wherein the word order sequence set is divided into a first word order sequence group and a second word order sequence group; calculating the difference between the first word order sequence in the first word order sequence group and the second word order sequence in the second word order sequence group; adjusting the prediction value of the first word order sequence in the first word order sequence group based on the difference; dividing at least part of the first word order sequence in the first word order sequence group to the second word order sequence group according to the adjusted prediction value of the first word order sequence in the first word order sequence group; and generating an expanded text comprising a corresponding second word order sequence based on the second word order sequence group. The difference between the expanded text and the reference text is improved.

Description

Technical Field

[0001] The embodiments described in this specification relate to the field of artificial intelligence, and specifically to a method, apparatus, device, and storage medium for generating text. Background Technology

[0002] Currently, dialogue models can identify the semantics of input text and provide response text. However, due to the diversity of language expression, text converging on the same semantics can have multiple expressions. Therefore, dialogue models require a large number of texts with different expressions converging on the same semantics for training. In existing technologies, annotators typically input multiple similar texts converging on the same semantics to form the training samples for the dialogue model.

[0003] Therefore, due to the limitations of the individual annotation habits of the annotators, the expression methods of similar texts with the same semantic meaning are relatively uniform. This leads to the problem of inaccurate semantic recognition in dialogue models trained on these texts. Summary of the Invention

[0004] In view of this, the embodiments of this specification aim to provide a method, apparatus, device, and storage medium for generating text. This can improve the differentiation between texts with similar semantics.

[0005] This specification provides a text generation method through multiple embodiments, the method comprising: obtaining a set of word order sequences including multiple word order sequences generated based on a reference text, and a prediction value of the word order sequences; wherein the prediction value is used to represent the generation probability of the word order sequences; wherein the set of word order sequences is divided into a first word order sequence group and a second word order sequence group; wherein the word order sequences included in the first word order sequence group are designated as first word order sequences, and the word order sequences included in the second word order sequence group are designated as second word order sequences; calculating the first word order sequence in the first word order sequence group and the second word order sequence group. The degree of difference between the second word order sequences in the first word order sequence group; wherein the degree of difference represents the degree of distinction between the first word order sequence and the second word order sequence; based on the degree of difference, the predicted value of the first word order sequence in the first word order sequence group is adjusted; according to the adjusted predicted value of the first word order sequence in the first word order sequence group, at least a portion of the first word order sequences in the first word order sequence group are assigned to the second word order sequence group as the second word order sequence in the second word order sequence group; based on the second word order sequence group, an extended text including the corresponding second word order sequence is generated; wherein the extended text has the same intent as the baseline text.

[0006] One embodiment of this specification provides a text generation apparatus, the apparatus comprising: an acquisition module, configured to acquire a set of word order sequences including multiple word order sequences generated based on a reference text, and a prediction value of the word order sequences; wherein the prediction value is used to represent the generation probability of the word order sequences; wherein the set of word order sequences is divided into a first word order sequence group and a second word order sequence group; wherein the word order sequences included in the first word order sequence group are designated as first word order sequences, and the word order sequences included in the second word order sequence group are designated as second word order sequences; and a calculation module, configured to calculate the first word order sequence in the first word order sequence group and the second word order sequence in the second word order sequence group. The system includes: a difference degree between word sequences; wherein the difference degree represents the degree of distinction between a first word sequence and a second word sequence; an adjustment module, configured to adjust the predicted value of a first word sequence in the first word sequence group based on the difference degree; a division module, configured to divide at least a portion of the first word sequences in the first word sequence group into a second word sequence group, as second word sequences in the second word sequence group, based on the adjusted predicted value of the first word sequences in the first word sequence group; and a generation module, configured to generate extended text including the corresponding second word sequences based on the second word sequence group; wherein the extended text has the same intent as the baseline text.

[0007] The various implementations provided in this specification divide the word sequence generated from the base text into a first word sequence and a second word sequence, and adjust the predicted value of the first word sequence based on the difference between the second word sequence and the first word sequence. Based on the predicted value, the first word sequence with a greater degree of difference from the second word sequence is selected, divided into the second word sequence, and extended text is generated based on the second word sequence, thereby improving the degree of difference between the generated extended text and the base text to a certain extent. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a schematic diagram of a text generation system provided for one embodiment of this specification.

[0010] Figure 2 This specification provides a flowchart illustrating a method for generating text according to one embodiment of the present invention.

[0011] Figure 3This is a flowchart illustrating a training method for a text generation model provided in one embodiment of this specification.

[0012] Figure 4 This is a schematic diagram of a text generation apparatus provided for one embodiment of this specification.

[0013] Figure 5 This is a schematic diagram of a training apparatus for a text generation model provided in one embodiment of this specification.

[0014] Figure 6 A schematic diagram of a computer device provided for one embodiment of this specification. Detailed Implementation

[0015] In related technologies, in order to ensure that the dialogue model can recognize texts that represent the same semantics but are expressed in different ways, training samples of the dialogue model can be constructed by using multiple texts that represent the same semantics but are expressed in different ways, so as to improve the dialogue model's ability to recognize text to a certain extent.

[0016] Therefore, the diversity of training samples has a certain impact on improving the text recognition ability of dialogue models.

[0017] In related technologies, annotators can construct similar texts with semantically similar but different expressions based on a baseline text, which can be used to build training samples for the dialogue model. However, due to the personal habits of the annotators, the differences between the similar texts constructed by the annotators are small, which is not conducive to using them as training samples for the dialogue model to improve the model's recognition ability.

[0018] Therefore, it is necessary to provide a text generation method to generate multiple benchmark word sequences based on a benchmark text, and to adjust the predicted values ​​of the word sequences according to the degree of difference between the word sequences, thereby solving the technical problem of small differences between the extended texts generated from the benchmark text.

[0019] This specification provides an example application scenario of a training system for a text generation model.

[0020] The training system for the text generation model can be used to train the text generation model. The text generation model can be an end-to-end model. Specifically, the text generation model can include an encoding network, a decoding network, and a text generation network. The text generation network can include dense layers and softmax layers. Optionally, the encoding network is a pre-trained language model based on BERT, and the decoding network is a pre-trained autoregressive language model based on the BERT structure. Of course, other models with encoding and decoding functions can also be used, and this application does not impose specific limitations on them. The text generation model can receive input text and output text that is semantically similar to the input text.

[0021] The training process of the text generation model may include the following steps.

[0022] The training system for the text generation model can pre-construct three types of training samples. The training system can use text from a historical text library to construct pre-training samples. Specifically, after segmenting historical text, the system randomly selects 20% of the segmentation results for masking to obtain an initial input word sequence. Furthermore, for the masked words, a corresponding initial label word sequence is constructed through inverse masking to form the pre-training samples. The initial input word sequence can serve as the input features of the pre-training text, and the initial label word sequence can serve as the labels of the pre-training text. The initial text generation model can be trained using these pre-training samples. Cross-entropy can be used to calculate the loss between the output of the initial text generation model and the initial label word sequence, and the loss is optimized using a weighted decaying Adam optimizer. The pre-training samples can also be a third type of training sample.

[0023] An initial text generation model trained using pre-trained samples can serve as a primary text generation model. The training system for the text generation model can construct intermediate training samples. Specifically, the intermediate training samples can include intermediate input text and intermediate label text. The intermediate label text can include dimensional text representing a semantic dimension of the intermediate input text and dimensional identifiers characterizing the dimensional text. Specifically, the dimensional identifiers can include intent identifiers representing the intent of the intermediate input text, entity identifiers representing entities within the intermediate input text, and question identifiers representing the problem expressed by the intermediate input text. Correspondingly, the dimensional text can include intent text representing the intended content of the intermediate input text, entity text representing entities involved in the intermediate input text, and question text representing the problem expressed by the intermediate input text. The corresponding dimensional identifiers and dimensional texts can form the intermediate label text. The training system for the text generation model can train the primary text generation model using the intermediate training samples, and use the trained primary text generation model as an intermediate text generation model. The intermediate training samples can be the first training samples.

[0024] The text generation model training system can also construct target training samples. These target training samples can include target input text and target label text; the target label text can express the same semantics as the target input text, but their text content is different. Specifically, the text generation model training system can pairwise group multiple texts representing the same semantic meaning to construct the target training samples. The intermediate text generation model is trained using these target training samples, and the trained intermediate model is used as the target text generation model to generate similar text.

[0025] This specification provides an example application scenario for a text generation system.

[0026] The text generation system can generate extended text that is semantically similar to the base text based on the base text.

[0027] The text generation system can first preprocess the baseline text. Specifically, the system can standardize the punctuation marks in the baseline text to standard punctuation marks, and then use a word segmenter to segment the baseline text into words, obtaining a baseline word sequence. Next, the system can input the baseline word sequence into a trained text generation model. This text generation model can include an encoder and a decoder.

[0028] The encoder of the text generation model generates an encoding vector based on the baseline word sequence. The encoding vector is input into the decoder to sequentially generate decoded words, forming a decoded word sequence, thereby generating the extended text. Specifically, during the generation of the extended text, the text generation system generates probability distributions for different subsequent decoded words based on the encoding vector and the preceding decoded words. The text generation system can select the top 50 decoded words with higher probabilities as candidate decoded words, and construct a candidate decoded word sequence based on the preceding decoded words. The predicted value corresponding to the candidate decoded word sequence can be determined based on the probability of the decoded words generated in different time sequences. The text generation system can select the candidate decoded word sequence with the highest predicted value and mark it as the target decoded word sequence. Then, the text generation system can calculate the difference between each candidate decoded word sequence not marked as the target decoded word sequence and the target decoded word sequence. The difference can be determined based on the proportion of non-repeating words in the candidate and target decoded word sequences among all words. Next, the text generation system can adjust the predicted values ​​of the candidate decoded word sequences based on the difference degree. Based on the adjusted predicted values, the candidate decoded word sequence with the highest predicted value can be determined again as the target decoded word sequence. Further, the difference degree between the remaining candidate decoded word sequences and multiple target decoded word sequences can be calculated separately, and the minimum difference degree among the multiple difference degrees corresponding to a candidate encoded word sequence can be determined as the target difference degree to adjust the predicted values ​​of the corresponding candidate encoded word sequences. The above steps are repeated until 20 of the 50 candidate decoded word sequences are selected as target decoded word sequences. Here, the target decoded word sequence can represent the second word sequence. Candidate decoded word sequences that are not selected as the second word sequence can be used as the first word sequence.

[0029] For words in different positions, the above steps can be used to generate and calculate the final decoded word sequence to form a sentence, thus generating extended text.

[0030] This specification provides a training system for a text generation model. The training system may include a client and a server. The client can be used to receive user-annotated training samples. The server can be used to train the text generation model.

[0031] Please see Figure 1 This specification provides a text generation system. The text generation system may include a server. The server can be used to execute the text generation method.

[0032] The client can be an electronic device with network access capabilities. Specifically, for example, the client can be a desktop computer, tablet computer, or laptop computer. Alternatively, the client can also be software that can run on the electronic device.

[0033] A server can be an electronic device with a certain computing power. It may include a network communication module, a processor, and a memory. Alternatively, the server can refer to software running on the electronic device. The server can also be a distributed server, a system with multiple processors, memory, network communication modules, etc., operating collaboratively. Or, the server can be a server cluster formed by several servers. Furthermore, with the development of science and technology, the server can also be a new technological means capable of realizing the corresponding functions of the embodiments described in the specification. For example, it could be a new form of "server" based on quantum computing.

[0034] Please see Figure 2 This description provides a method for generating text. This text generation method can be applied to a server. The text generation method may include the following steps.

[0035] Step S101: Obtain a set of word order sequences including multiple word order sequences generated based on the reference text, and a predicted value of the word order sequences; wherein the predicted value is used to represent the generation probability of the word order sequence; wherein the set of word order sequences is divided into a first word order sequence group and a second word order sequence group; wherein the word order sequences included in the first word order sequence group are used as the first word order sequence, and the word order sequences included in the second word order sequence group are used as the second word order sequence.

[0036] In some cases, multiple word sequences can be generated from a baseline text, forming a word sequence set. A target word sequence can be determined from this set to generate extended text. The extended text can express semantics that are close to those of the baseline text. Extended text with semantically similar meaning to the baseline text can be used as training samples for the model, thereby improving the model's ability to recognize the semantics of texts with different expressions to a certain extent.

[0037] The base text can represent text containing base information for generating the extended text. Specifically, corresponding extended text can be generated based on the base text. The semantics represented by the extended text can be similar to those represented by the base text. For example, the base text could be "how to apply for a membership card". Correspondingly, the extended text could include "methods for applying for a membership card" and "I want to apply for a membership card", etc. Accordingly, the dialogue model can be trained using the extended text, enabling the dialogue model to learn different expressions of the same semantics.

[0038] The word sequence may include multiple words with an ordered relationship. Alternatively, the word sequence may contain only one word. The word sequence may be generated based on the base text. Specifically, the word sequence may be constructed by sequentially generating words based on information from the base text.

[0039] The word sequence can consist of at least one word. Each word can represent a vocabulary or word. For example, the word sequence can be formed by the words "today" and "weather". Of course, the word can also be a single character. In some embodiments, the word can also be a word vector representing different vocabulary or words, or a text vector with a certain semantic meaning.

[0040] In some implementations, the word order sequence can be a sequence composed of multiple word orders predicted from a reference text. Specifically, encoding the reference text yields an encoding vector. The encoding vector can represent the semantic information of the reference text. The word order sequence can be generated based on the encoding vector. For example, inputting the encoding vector and previously predicted word orders into the decoding network of a text generation model can sequentially generate subsequently predicted decoded word orders, and the word order sequence can be formed based on the decoded word orders. For example, based on the encoding vector and the preceding word orders "BOS" and "today", the next word order can be predicted. The predicted next word order can have multiple candidate word orders. The candidate word orders can correspond to predicted values. Based on the predicted values, the target word order can be determined from the candidate word orders. For example, the predicted value of "weather" is the highest. Therefore, "weather" can be used as the next word order of "today". Correspondingly, the predicted value of the word order sequence can be determined based on the word order prediction probability of each word order. For example, the threshold for "today" is 0.8, and the threshold for "weather" is 0.5. The predicted value for the word sequence ["Today", "Weather"] can be 0.8 plus 0.5 divided by 2, resulting in 0.65. This predicted value can represent the generation probability of the corresponding word or word sequence. The generation probability can represent the degree of matching between the text information represented by the word sequence and the baseline text. In some embodiments, the generation probability can also represent the likelihood of the extended text represented by the word sequence appearing in a specific task.

[0041] In some implementations, the word sequence may include only a portion of the words constituting the extended text. For example, the extended text composed of the final predicted word sequence may be "The weather is really nice today". Correspondingly, the word sequence generated during the generation of the extended text may be ["Today"], or ["Today", "Weather"], or of course, ["Today", "Weather", "Nice"]. Based on the word sequence, further words can be predicted and added to the end of the word sequence to form a new word sequence. If the word at the end is a closing word, the extended text can be generated based on the corresponding word sequence.

[0042] The set of word sequences can be divided into a first word sequence group and a second word sequence group. Correspondingly, the word sequences can be divided into a first word sequence and a second word sequence. The first word sequence can belong to the first word sequence group. The second word sequence can belong to the second word sequence group. In some cases, the predicted value of the second word sequence can be relatively high. The second word sequence in the second word sequence group can be used as a target word sequence to generate the extended text. In some embodiments, the process of dividing the set of word sequences into the first word sequence group and the second word sequence group can be based on the predicted values ​​of the word sequences in the set. For example, word sequences with predicted values ​​higher than a specified threshold can be used as second word sequences and added to the second word sequence group. Word sequences that are not added to the second word sequence group can be used as first word sequences and added to the first word sequence group.

[0043] A method for receiving a set of word sequences, including multiple word sequences, generated from a reference text, and predicted values ​​of the word sequences, can be achieved by a module in a text generation system providing the set of word sequences to a server executing the text generation method. Alternatively, the server can receive the set of word sequences after generating them based on a decoder in a text generation model.

[0044] Step S102: Calculate the difference between the first word sequence in the first word sequence group and the second word sequence in the second word sequence group; wherein the difference represents the degree of distinction between the first word sequence and the second word sequence.

[0045] In some cases, the target word sequence can be determined from the word sequence based on the predicted value to generate extended text. However, the difference between multiple word sequences with high predicted values ​​may be small. Consequently, the difference in the extended text generated based on the word sequence will also be small, and it will be impossible to obtain extended text with significant differences in expression and similar semantics. For example, the word sequences ["today", "weather", "great"] and ["weather", "great", "today"] both have high predicted values, but their expressions are very similar. Therefore, if the model is trained based on extended text with similar expressions, the model may only learn relatively similar expressions. As a result, the ability to recognize the semantics of texts with different expressions of the same meaning is poor, which may weaken the model's ability to recognize texts with different expressions to some extent.

[0046] The degree of difference can represent the degree of distinction between the text content of the first word sequence and the second word sequence. Specifically, for example, if the first and second word sequences include multiple identical words, it indicates a relatively small degree of difference between them. For instance, the first word sequence ["today", "weather", "great"] and the second word sequence ["today", "weather", "good"] include the words "today", "weather", "great", and "good". Correspondingly, the first and second word sequences include two identical words. Conversely, if the first and second word sequences do not contain any identical words, it indicates a relatively large degree of difference between them. The degree of difference can be represented by the reciprocal of the number of identical words between the first and second word sequences. Alternatively, the degree of difference can be expressed as the ratio of the number of different words between the first and second word sequences to the total number of words. In some implementations, the difference between the first word sequence and the second word sequence can also be calculated by calculating the Euclidean distance between the vectors represented by the first word sequence and the second word sequence, respectively.

[0047] Step S103: Based on the difference degree, adjust the predicted value of the first word sequence in the first word sequence group.

[0048] In some cases, to generate multiple extended texts with similar semantics but different content, ensuring that the generated extended texts cover a wider range of language expression habits, a second word sequence can be selected based on the predicted value during the generation of the word sequence included in the extended text. Furthermore, the predicted value of the first word sequence can be adjusted based on the degree of difference between the first and second word sequences to select the target word sequence from the first word sequence.

[0049] The difference degree can be used to represent the degree of difference between the text content of the first word sequence and the second word sequence. The difference degree can be determined by calculating the Euclidean distance between the representation vectors of the first word sequence and the second word sequence, and then further calculating the difference degree. Alternatively, the difference degree can be calculated as the ratio between the number of identical words and the number of different words in the first and second word sequences.

[0050] In some implementations, the degree of difference may also represent the similarity between the text content of the first word sequence and the second word sequence.

[0051] The method for adjusting the predicted value of the first word sequence based on the difference degree and the predicted value of the first word sequence can be to add the difference degree to the predicted value of the first word sequence to obtain the adjusted predicted value. Alternatively, the method can be to multiply the predicted value of the first word sequence by the difference degree to obtain the adjusted predicted value.

[0052] Accordingly, based on the difference between the first and second word sequences and the predicted value of the first word sequence, the predicted value of the first word sequence can be adjusted. Based on the adjusted predicted value, the first word sequence in the first word sequence group can be further classified as the second word sequence and assigned to the second word sequence group. Further, based on the difference between the second word sequence in the updated second word sequence group and the first word sequence in the first word sequence group, the first word sequence can be further classified as the second word sequence and assigned to the second word sequence group. If the number of second word sequences in the second word sequence group meets a threshold, the second word sequence in the second word sequence group can be used as the target word sequence. Text can be generated based on the target word sequence.

[0053] Step S104: Based on the predicted value of the first word sequence in the adjusted first word sequence group, at least a portion of the first word sequence in the first word sequence group is assigned to the second word sequence group as the second word sequence in the second word sequence group.

[0054] In some cases, based on the predicted value of the adjusted first word order sequence, the corresponding first word order sequence can be divided into a second word order sequence to generate extended text based on the second word order sequence.

[0055] In some cases, the second word sequence can be a target word sequence. The predicted value of the first word sequence is adjusted based on the difference between the first and second word sequences to further identify more target word sequences within the first word sequence. Adjusting the predicted value by the difference allows for the identification of multiple target word sequences based on task requirements. For example, if the task requirement is to maintain a high level of difference between the multiple target word sequences, the difference can be added to the predicted value to adjust the predicted value of the first word sequence, increasing the probability that the first word sequence with a higher difference from the second word sequence will be assigned to the second word sequence group. Alternatively, the task requirement could be to maintain a low level of difference between the target word sequences; in this case, the predicted value can be subtracted from the difference.

[0056] Step S105: Generate extended text including the corresponding second word sequence based on the second word sequence group; wherein the extended text has the same intent as the base text.

[0057] In some cases, extended text can be further generated based on the second word order sequence in the second word order sequence group.

[0058] A method for generating extended text including corresponding second command sequences based on the second command sequence group can be to update the second command sequence group based on the second command sequences in the second command sequence group respectively. The extended text is then generated based on the updated second command sequence group. Specifically, for example, the second command sequence group can include second command sequences such as ["BOS", "how"] and ["BOS", "how"]. The next command can be predicted based on existing commands in the second command sequence, thus updating the second command sequence. The next predicted command for ["BOS", "how"] could be "handle". A predicted command for ["BOS", "how"] could be "use". Correspondingly, the updated second command sequence group can include ["BOS", "how", "handle"] and ["BOS", "how", "use"]. If the next predicted command is a terminating command, extended text can be generated based on the command sequence. For example, a terminating command can be represented by "EOS". The extended text corresponding to ["BOS", "how", "handle", "member card", "EOS"] is "How to handle a member card".

[0059] In some implementations, the method for updating the second word order sequence group to further generate extended text can be to predict multiple next-position word orders for each second word order sequence in the second word order sequence group, obtaining multiple candidate word order sequences. Each candidate word order sequence corresponds to a predicted value. Further, at least a portion of the candidate word order sequences can be selected as the updated second word order sequences to update the second word order sequence group. Extended text can be further generated based on the updated second word order sequence group. Specifically, the method for selecting at least a portion of the candidate word order sequences as the updated second word order sequences to update the second word order sequence group can be to treat the multiple candidate word order sequences as the word order sequence set. Further, the word order sequences in the word order sequence set are divided into first word order sequences and second word order sequences. Further, steps S102 to S104 are executed, and the finally determined second word order sequence group is used as the updated second word order sequence group.

[0060] The statement that the extended text and the base text have the same intent can mean that the semantics of the extended text and the base text tend to be the same. That is, the extended text and the base text express the same semantics. Of course, in some embodiments, "same intent" can also mean that the semantic intent expressed by the base text is the same as the semantic intent expressed by the extended text.

[0061] In some embodiments, the step of allocating at least a portion of the first word sequence in the first word sequence group to the second word sequence group as second word sequences, based on the predicted values ​​of the first word sequences in the adjusted first word sequence group, includes: allocating first word sequences in the adjusted first word sequence group whose predicted values ​​exceed a first threshold to the second word sequence group as second word sequences of the second word sequence group. In some cases, after each adjustment of the predicted values ​​of the first word sequence, at least a portion of the first word sequence can be allocated to the second word sequence group as second word sequences of the second word sequence group based on the adjusted predicted values ​​of the first word sequence. Specifically, the first word sequences in the adjusted first word sequence group whose predicted values ​​exceed a first threshold can be allocated to the second word sequence group as second word sequences of the second word sequence group. The first threshold can be a preset threshold. Alternatively, the first threshold can be the maximum value of the adjusted predicted values ​​of the first word sequence.

[0062] In some embodiments, the word order sequence set has a plurality of second word order sequences; the step of calculating the difference between the first word order sequence in the first word order sequence group and the second word order sequence included in the second word order sequence group includes: calculating the difference between each first word order sequence in the first word order sequence group and the second word order sequence in the second word order sequence group respectively; determining a target difference in the difference corresponding to each first word order sequence respectively; the step of adjusting the predicted value of the first word order sequence in the first word order sequence group based on the difference includes: adjusting the predicted value of the corresponding first word order sequence based on the target difference of each first word order sequence.

[0063] In some cases, the second instruction sequence group may include multiple second instruction sequences. The first instruction sequence can calculate its difference with each of the multiple second instruction sequences. Specifically, the first instruction sequence can calculate its difference with each of the second instruction sequences in the second instruction sequence group, resulting in multiple difference scores. Among these multiple difference scores, a target difference score can be determined to further adjust the predicted value based on the target difference score. Specifically, the difference score between each pair of instruction sequences can be calculated, from the first instruction sequence in the first instruction sequence group to the second instruction sequence in the second instruction sequence group. Further, based on the difference scores calculated between the first instruction sequence and the multiple second instruction sequences, the second instruction sequence corresponding to the first instruction sequence can be determined.

[0064] In some implementations, the difference between each first word sequence in the first word sequence group and each second word sequence in the second word sequence can be calculated to determine the target difference for the first word sequence. Alternatively, the difference between each first word sequence in the first word sequence group and some second word sequences in the second word sequence group can also be calculated to determine the target difference. In some implementations, the difference can also be calculated between each first word sequence in the first word sequence group and one second word sequence in the second word sequence group. Here, the second word sequence can be a second word sequence in the second word sequence group whose predicted value meets a specified condition.

[0065] The method for determining the target difference degree among the difference degrees corresponding to each first word sequence can be determined according to task requirements. For example, to ensure a large degree of difference among the generated multiple word sequences, the difference degree with the smallest value among the difference degrees can be selected as the target difference degree. The smallest value of the difference degree can be expressed as the small difference between the first word sequence and the corresponding second word sequence. Therefore, changing the first word sequence to the second word sequence may result in a small difference degree among the second word sequences. Therefore, in the process of adjusting the predicted value, the smallest value of the difference degree can be selected as the target difference degree. Furthermore, the target difference degree and the predicted value can be calculated by weighting according to a preset weight. This ensures that the predicted value corresponding to the first word sequence with a low target difference degree is also relatively small. In some embodiments, to ensure a small degree of difference among the generated multiple word sequences, the maximum difference degree can also be selected as the target difference degree.

[0066] In the task of generating extended text, in order to ensure that the semantics of the extended text tend to be consistent, but the text content can have large differences to cover more language expressions to a certain extent, the minimum value of difference can be selected as the target difference.

[0067] In some implementations, the step of determining the target difference in the difference corresponding to each first word sequence includes: determining the difference corresponding to each first word sequence with a value lower than a second threshold as the target difference.

[0068] In some cases, to ensure that a first word sequence, after adjustment, has a higher probability of being classified into the second word sequence group if it has a greater degree of difference from all second word sequences in the second word sequence group, the degree of difference for each first word sequence that is lower than a second threshold can be determined as the target degree of difference. Furthermore, based on the target degree of difference, the predicted value of the first word sequence can be adjusted.

[0069] In some implementations, the step of adjusting the predicted value of the corresponding first word sequence based on the target difference of each first word sequence includes adjusting the predicted value of the first word sequence by weighted summing of the predicted value of the first word sequence and the target difference corresponding to the first word sequence to obtain the adjusted predicted value of the first word sequence.

[0070] By performing a weighted sum of the predicted value of the first token sequence and the target difference corresponding to the second token sequence, the predicted value of the first token sequence can be adjusted. Among them, the predicted value of the first token sequence with a larger difference from the token sequences in the second token sequence group has a larger bonus quantity. Correspondingly, the first token sequence with a larger difference from the token sequences in the second token sequence group is added to the second token sequence group as the second token sequence of the second token sequence group, and the possibility of generating an extended text based on the second token group increases. To a certain extent, the probability of generating an extended text with a larger difference can be increased to a certain extent.

[0071] In some embodiments, the token sequence includes at least one token; the step of calculating the difference degree between the first token sequence and the second token sequence may include: calculating the difference degree according to the ratio of the number of non-repeating tokens between the first token sequence and the second token sequence to the total number of tokens.

[0072] In some cases, for the difference between the output text contents, the difference degree may be calculated according to the ratio of the number of non-repeating tokens between the first token sequence and the second token sequence to the total number of tokens.

[0073] The token may represent a text unit. For example, the token may represent a character or a word, etc. Specifically, for example, each token may be composed of a single Chinese character. The first token sequence may be ["今", "日", "天", "气", "好"], and the second token sequence may be ["今", "天", "气", "不", "错"]. Correspondingly, the non-repeating tokens between the first token sequence and the second token sequence may include: "日", "好", "不", and "错". The total tokens of the first token sequence and the second token sequence may include: "今", "日", "天", "气", "好", "不", "错". Among them, the number of identical tokens between the first token sequence and the second token sequence may be 4. The number of different tokens between the first token sequence and the second token sequence may be 7. Correspondingly, the difference degree may be 4 / 7.

[0074] In some embodiments, a multi-token phrase formed based on a specified number of adjacent tokens; the method further includes: calculating the ratio of the number of non-repeating multi-token phrases between the first token sequence and the second token sequence to the total number of multi-token phrases; the step of calculating the difference degree according to the ratio of the number of non-repeating tokens between the first token sequence and the second token sequence to the total number of tokens includes: performing a weighted sum of the ratio of the number of non-repeating multi-token phrases to the total number of multi-token phrases and the ratio of the number of non-repeating tokens to the total number of tokens to obtain the difference degree.

[0075] In some implementations, the words in the first word sequence and the second word sequence can be divided into multiple categories. Based on the different categories of words, a sub-difference degree can be obtained by calculating the ratio between the number of identical words and the number of different words included in the first and second word sequences within the respective categories. Then, a weighted sum of the sub-difference degrees for different categories can be performed to determine the total difference degree. The word categories can be divided based on parts of speech or the number of characters included in the word sequence.

[0076] Specifically, for example, the degree of difference can be determined by the proportion of non-repeating unary words and the proportion of non-repeating multi-word groups in the first and second word sequences. The multi-word group can be formed by multiple adjacent words. Specifically, the multiple adjacent words can constitute a word group. The multi-word group can be a bigram.

[0077] Specifically, the degree of difference can be determined by Formula 1.

[0078]

[0079] s0 can represent a first word sequence. s1 can represent a second word sequence. d(s0,s1) can represent the difference between the first word sequence and the second word sequence.

[0080] Wherein, Unique1(s0,s1) can represent the number of unary words that are not repeated between the first word sequence and the second word sequence, and Combine1(s0,s1) can represent the total number of unary words in the first word sequence and the second word sequence. Unique2(s0,s1) can represent the number of bigrams that are not repeated between the first word sequence and the second word sequence, and Combine2(s0,s1) can represent the total number of bigrams in the first word sequence and the second word sequence.

[0081] In some implementations, to ensure the semantic differences between the output texts, the similarity between vectors that represent the semantics of the first word sequence and the second word sequence can be used to characterize them.

[0082] Please see Figure 3 This specification provides a method for training a text generation model, one embodiment of which can be applied to a server. The method may include the following steps.

[0083] Step S201: Obtain a first training sample; wherein the first training sample includes a first input text and a first label text; the first label text includes a dimension identifier representing a semantic dimension, and a dimension text representing the content of the semantic dimension expressed by the first input text; wherein the semantic dimension represents the category into which the first input text is divided according to the expressed semantics.

[0084] In some cases, training samples can be constructed for training the text generation model before training the text generation model, so that the text generation model can learn the information covered by the training samples.

[0085] In some cases, text generation models can generate extended text that expresses semantics similar to the baseline text based on the input text. In some implementations, the extended text may also be referred to as similar text.

[0086] In some cases, the baseline text and corresponding extended text can be used to train the dialogue model. Because language has rich expressive forms, extended text that shares similar semantics with the output text can be generated based on the input text and used to train the dialogue model. This can improve the generalization performance of the dialogue model to some extent.

[0087] Furthermore, to ensure the performance of the dialogue model to a certain extent and to avoid semantic discrepancies caused by the diversity of language expressions, the extended text generated by the text generation model should maintain consistency with the corresponding semantic dimensions expressed by the input text as much as possible. Therefore, during the training of the text generation model, labeled training samples can be used to train the model, enabling it to learn the ability to recognize the semantic dimensions of text to a certain extent.

[0088] The first training sample can be used to train the primary text generation model. The primary text generation model can be a pre-trained model. The first training sample can include a first input text and a first label text. The first input text can be used as the input to the primary text generation model, and the first label text can be used as the target output of the primary text generation model.

[0089] The first input text can be derived from text in a library. The first input text can undergo text preprocessing. For example, the first input text can undergo punctuation normalization or word segmentation.

[0090] The first tag text may be a dimension text that includes content representing at least one semantic dimension of the first input text, and a dimension identifier for characterizing the semantic dimension.

[0091] The semantic dimension can represent the category into which the first input text is classified according to the semantics it expresses. Specifically, the semantic information expressed by the first input text can be of various types. The category of semantic information can be the semantic dimension. For example, the first input text can be "how to apply for a membership card". Correspondingly, the semantic information expressed by the first input text can include intent, entity, etc. Correspondingly, the semantic dimension can be the intent expressed by the text and the entity involved in the text. Wherein, the intent can be "apply for membership". The entity can be "membership card".

[0092] Methods for obtaining the first training sample may include receiving manually labeled first training samples or reading the first sample from a database.

[0093] Step S202: Train the primary text generation model based on the first training sample to obtain the intermediate text generation model.

[0094] In some cases, a primary text generation model can be trained based on the first training samples obtained from the construction.

[0095] The primary text generation model can be a pre-trained model. The primary model can be pre-trained using an autoencoder method. The intermediate text generation model can be obtained by further training the primary text generation model using the first training samples.

[0096] The process of training an intermediate text generation model based on a primary text generation model using a first training sample pair can be achieved by training the primary text generation model using the first training samples. If the primary text generation model tends to fit the first training samples well, it can be used as the intermediate text generation model. In some implementations, a loss function can be constructed using cross-entropy loss and optimized using the Adam optimizer.

[0097] In some implementations, the process of training an intermediate text generation model based on a primary text generation model using a first training sample pair can be as follows: First, a word segmenter can be used to segment the first input text and the first label text of the first training sample, resulting in a sample word order sequence. Next, word vectors can be used to encode the sample word order sequence. The encoded word order sequence is then input into the primary text generation model to obtain a first language representation vector. Based on the first language representation vector, the difference between the output decoded sequence and the label word order sequence is predicted. Specifically, the cross-entropy between the label word order sequence and the decoded word order sequence can be calculated, and a second loss can be minimized using a weighted decaying Adam optimizer to obtain the intermediate text generation model.

[0098] Step S203: Based on the intermediate text generation model, generate a target text generation model; wherein the output text of the target text generation model and the input text of the target text generation model express the same semantics.

[0099] In some cases, to enable the target text generation model to generate extended text that is semantically similar to the baseline text, the parameters of the intermediate text generation model can be further adjusted. This allows the adjusted target text generation model to generate extended text that is semantically similar to the baseline text, while still possessing semantic recognition capabilities.

[0100] The extended text output by the target text generation model is semantically similar to the base text input to the target text generation model. The target text generation model can be obtained by further training on corresponding training samples based on the intermediate text generation model. The structure of the target text generation model can be the same as that of the intermediate text generation model. In some embodiments, the target text generation model can also utilize some parameters of the intermediate text generation model and further adjust the model parameters to obtain the target text generation model. For example, transfer learning can be used to initialize the target text generation model by selecting the encoder parameters and the decoder parameters (excluding the output layer) of the intermediate text generation model. Based on this, the final target text generation model is obtained through training on corresponding training samples.

[0101] In some implementations, the step of generating a target text generation model based on the intermediate text generation model may include: training the intermediate text generation model using a second training sample to obtain the target text generation model; wherein the second training sample includes a second input text and a second label text; the second label text and the second input text express similar semantics, but have different text content.

[0102] In some cases, to enable the intermediate text generation model to generate the target text generation model, the intermediate text generation model can be trained with a second training sample to adjust its parameters. This allows the model to generate output text that is semantically similar to the input text, thus obtaining the target text generation model.

[0103] In some implementations, the method for constructing the second training sample may involve first obtaining a set of similar texts. This set may include multiple similar texts, which tend to have similar semantics. Then, the similar texts in the set are grouped into pairs to form similar text pairs, which can be used as the second input text and the second label text of the second training sample, respectively.

[0104] The second training sample may include a second input text and a second labeled text; the second labeled text and the second input text express similar semantics, but their text content is different. Specifically, for example, the second input text may be "how to apply for a membership card". Correspondingly, the second labeled text may be "methods for applying for a membership card". Here, "how to apply for a membership card" and "methods for applying for a membership card" have similar semantics. In some embodiments, the second training sample may also be text after word segmentation.

[0105] The process of training the intermediate text generation model using the second training samples to obtain the target text generation model can be achieved by constructing a loss function and then optimizing the loss using an optimizer. When the loss tends to be constant, or when a preset condition is met, the intermediate text generation model after training is used as the target text generation model. Specifically, the second input text and the second label text can be segmented using a word segmenter, and further encoded using word vectors to obtain an input word sequence and a label word sequence. The intermediate text generation model of the input word sequence is used to obtain a second language representation vector. The cross-entropy loss between the second language representation vector and the label word sequence is calculated as a third loss, and the target text generation model is obtained by minimizing the third loss using a weighted decaying Adam optimizer.

[0106] In some embodiments, the training method of the text generation model may further include: training the initial text generation model with a third training sample to obtain the primary text generation model; wherein the third training sample includes a third input text and a third label text; the third input text includes mask words; and the third label text includes label words corresponding to the mask words.

[0107] In some cases, the text generated by the text generation model may not perfectly conform to human language habits. For example, the text generated by the model might be "registered in a membership card manner." Therefore, the text generation model can be pre-trained using text from a corpus to enable it to master language expression. Furthermore, since there is a large amount of unlabeled text data in the corpus, input and output text can be constructed using masks and inverse masks. This allows the text generation model to learn independently, improving corpus utilization and reducing manual annotation costs to some extent.

[0108] The third training sample includes a third input text and a third label text; the third input text includes masked words; the third label text includes the label words corresponding to the masked words.

[0109] The masked vocabulary can represent the words to be replaced using a specified mask. Correspondingly, the tag vocabulary can represent the words before the masked vocabulary is replaced. Specifically, for example, the input text can be "The weather is really nice today". After word segmentation, the input text can be represented as ["Today", "Weather", "Really nice"]. During the construction of the third training sample, some words in the input text can be randomly masked. The masking process can include replacing with a specified mask identifier, replacing with other words, or leaving it unchanged. Correspondingly, the third input text of the third training sample corresponding to the input text can be represented as ["Today", "[MASK]", "Really nice"]. Correspondingly, the third tag text can be represented as ["Weather"]. The third tag text can be used to indicate the truth value of the masked words. Of course, the third tag text can also include the position of the masked words in the sentence. In some embodiments, the third input text and the third tag text can also include multiple placeholders to ensure that the number of inputs in the input text and the number of inputs in the output text are the same in time. For example, the third input text can be represented as ["Today", "[MASK]", "Great", "[PAD]"]. The third label text can be represented as ["[PAD]", "Weather", "[PADDING]", "[PAD]"].

[0110] The third training sample allows the initial text generation model to learn more linguistic expression capabilities. Specifically, the initial text generation model can represent a model with randomly initialized parameters, or a model with optimized parameters. The method of training the initial text generation model using the third training sample to obtain the primary text generation model can also be achieved by using methods such as the cross-entropy loss function and the Adam optimizer. This ensures, to a certain extent, that the intermediate text generation model and the target text generation model possess superior semantic recognition and expression capabilities.

[0111] In some implementations, the semantic dimension includes the intent expressed by the first input text; correspondingly, the first tag text includes dimensional text representing the content of the intent, and dimensional identifiers for characterizing the intent; or, the semantic dimension includes the entity involved in the first input text; correspondingly, the first tag text includes dimensional text representing the entity, and dimensional identifiers for characterizing the entity; or, the semantic dimension includes the question answered by the first input text; correspondingly, the first tag text includes dimensional text representing the content of the question, and dimensional identifiers for characterizing the question.

[0112] In some cases, the first training sample can be divided into training samples corresponding to different semantic dimensions, so that the model can, to a certain extent, have the ability to recognize different semantic dimensions.

[0113] Specifically, the first training sample may include a first label text whose semantic dimension is intent. For example, the first input text of the first training sample may be "How to apply for a membership card". Correspondingly, the first label text may be "Intent: Apply for membership". Here, "Apply for membership" in the first label text can represent the content of the intent. The "intent" in the first label text can be the dimension identifier used to characterize the semantic dimension of "Apply for membership" as intent.

[0114] The first training sample may include first label text with the semantic dimension of an entity. For example, the first input text of the first training sample may be "How to apply for a membership card". Correspondingly, the first label text may be "Entity: Membership Card". Here, "Membership Card" in the first label text can represent an entity object. The "entity" in the first label text may be the dimension identifier used to characterize that the semantic dimension represented by "Membership Card" is an entity.

[0115] The first training sample may include a first label text with the semantic dimension of a question. Correspondingly, the first input text may be the answer to the question. For example, the first input text of the first training sample may be "Please process the application in APP->My->Member Card". Correspondingly, the first label text may be "Question: How to process a member card". Here, "How to process a member card" in the first label text can represent the content of the question. The "question" in the first label text can be the dimension identifier, used to characterize the semantic dimension of "How to process a member card" as a question. Correspondingly, the first input text "Please process the application in APP->My->Member Card" can represent the answer to the question. By using the answer to the question as the first input text and adding the question content to the first label text, the intermediate text generation model can better focus on the answer to the question, thereby improving its ability to recognize questions and answers to a certain extent. Furthermore, generating the target text generation model based on the ability to recognize questions and answers allows the text generated by the target text generation model to have a certain logical reasoning ability and semantic understanding ability. Correspondingly, using the text generated by the target text generation model to train a dialogue model is more conducive to improving the performance of the dialogue model.

[0116] In some implementations, the dimension identifier may not be limited to strings. The dimension identifier may also be a number or a specified symbol.

[0117] Please see Figure 4 This specification provides a text generation apparatus according to one embodiment. The text generation apparatus may include an acquisition module, an adjustment module, a calculation module, a segmentation module, and a generation module.

[0118] A receiving module is used to receive a set of word order sequences generated based on a reference text; wherein the set of word order sequences includes at least a plurality of word order sequences; wherein each word order sequence corresponds to a predicted value representing the probability of reasonableness of the word order sequence; the word order sequences are divided into a first word order sequence and a second word order sequence.

[0119] A calculation module is used to calculate the difference degree between the first word sequence and the second word sequence; wherein the difference degree represents the degree of difference between the text content of the first word sequence and the second word sequence.

[0120] An adjustment module is used to adjust the predicted value of the first word sequence based on the difference degree and the predicted value of the first word sequence.

[0121] A generation module is used to select a word sequence from a set of word sequences based on the predicted value to generate an extended text that includes the word sequence; wherein the extended text expresses the same semantics as the baseline text.

[0122] Please see Figure 5 One embodiment of this specification provides a training apparatus for a text generation model. The training apparatus for the text generation model may include an acquisition module, a training module, and a generation module.

[0123] An acquisition module is used to acquire a first training sample; wherein the first training sample includes a first input text and a first label text; the first label text includes a dimension identifier representing a semantic dimension, and a dimension text representing the content of the semantic dimension expressed by the first input text; wherein the semantic dimension represents the category into which the first input text is divided according to the expressed semantics.

[0124] The training module is used to train the primary text generation model based on the first training samples to obtain the intermediate text generation model.

[0125] The generation module is used to generate a target text generation model based on the intermediate text generation model; wherein the output text of the target text generation model and the input text of the target text generation model express the same semantics.

[0126] Please see Figure 6 One embodiment of the specification provides a computer device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor, when executing the computer program, implements the training method and the text generation method of any of the above-described text generation models.

[0127] This specification also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer, causes the computer to perform the training method and text generation method of the text generation model in any of the above embodiments.

[0128] This specification also provides a computer program product containing instructions that, when executed by a computer, cause the computer to perform the training method and text generation method of the text generation model in any of the above embodiments.

[0129] It is understood that the specific examples in this document are only intended to help those skilled in the art better understand the embodiments described herein, and are not intended to limit the scope of the invention.

[0130] It is understood that in the various embodiments described in this specification, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments described in this specification.

[0131] It is understood that the various implementation methods described in this specification can be implemented individually or in combination, and the implementation methods in this specification are not limited in this respect.

[0132] Unless otherwise stated, all technical and scientific terms used in the embodiments of this specification have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this specification. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items. The singular forms "a," "the," and "the" as used in the embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0133] It is understood that the processor in the embodiments of this specification can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiments can be completed by the integrated logic circuits in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this specification. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this specification can be directly implemented by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory; the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above methods.

[0134] It is understood that the memory in the embodiments of this specification may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM). It should be noted that the memory in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0135] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this specification.

[0136] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the aforementioned method implementations, and will not be repeated here.

[0137] In the several embodiments provided in this specification, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0138] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0139] In addition, the functional units in the various embodiments of this specification can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0140] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of this specification, in essence, or the parts that contribute to the prior art, or parts of the technical solutions, can be embodied in the form of software products. These computer software products are stored in a storage medium and include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this specification. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0141] The above description is merely a specific embodiment of this specification, but the scope of protection of this invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this specification should be included within the scope of protection of this specification. Therefore, the scope of protection of this invention should be determined by the scope of the claims.

Claims

1. A method for generating text, characterized in that, The method includes: Obtain a set of word order sequences generated from a reference text, including multiple word order sequences, and a predicted value for the word order sequences; wherein the predicted value is used to represent the generation probability of the word order sequence; wherein the set of word order sequences is divided into a first word order sequence group and a second word order sequence group; wherein the word order sequences included in the first word order sequence group are designated as the first word order sequence, and the word order sequences included in the second word order sequence group are designated as the second word order sequence; Calculate the difference between the first word sequence in the first word sequence group and the second word sequence in the second word sequence group; wherein the difference represents the degree of distinction between the first word sequence and the second word sequence. Based on the difference, adjust the predicted value of the first word sequence in the first word sequence group; Based on the predicted value of the first word sequence in the adjusted first word sequence group, at least a portion of the first word sequence in the first word sequence group is assigned to the second word sequence group as the second word sequence in the second word sequence group; An extended text comprising the corresponding second word order sequence is generated based on the second word order sequence group; wherein the extended text has the same intent as the base text.

2. The method according to claim 1, characterized in that, The step of allocating at least a portion of the first word sequences in the first word sequence group to the second word sequence group as second word sequences, based on the predicted values ​​of the first word sequences in the adjusted first word sequence group, includes: The first word sequence in the first word sequence group whose predicted value exceeds the first threshold after adjustment is assigned to the second word sequence group, and is used as the second word sequence of the second word sequence group.

3. The method according to claim 1, characterized in that, The step of calculating the difference between the first word sequence in the first word sequence group and the second word sequence in the second word sequence group includes: Calculate the difference between each first word sequence in the first word sequence group and the second word sequence in the second word sequence group; Determine the target difference in the difference score corresponding to each first word sequence; The step of adjusting the predicted value of the first word sequence in the first word sequence group based on the difference degree includes: adjusting the predicted value of the corresponding first word sequence based on the target difference degree of each first word sequence.

4. The method according to claim 3, characterized in that, The steps for determining the target dissimilarity in the dissimilarity corresponding to each first word sequence include: The difference between each first word and the corresponding sequence value below the second threshold is determined as the target difference.

5. The method according to claim 3, characterized in that, In the step of adjusting the predicted value of the corresponding first word sequence based on the target dissimilarity of each first word sequence, the methods for adjusting the predicted value of the first word sequence include: The predicted value of the first word order sequence is weighted and summed with the target difference corresponding to the first word order sequence to obtain the adjusted predicted value of the first word order sequence.

6. The method according to claim 1, characterized in that, The word order sequence includes at least one word order; the step of calculating the difference between the first word order sequence in the first word order sequence group and the second word order sequence included in the second word order sequence group includes: The degree of difference is calculated based on the ratio of the number of non-repeating words between the first and second word sequences to the total number of words.

7. The method according to claim 6, characterized in that, The method further includes: multi-word groups formed based on a specified number of adjacent words; Calculate the ratio of the number of non-repeating multi-word phrases to the total number of multi-word phrases between the first and second word sequences; The step of calculating the difference degree based on the ratio of the number of non-repeating words between the first word sequence and the second word sequence to the total number of words includes: weighted summing of the ratio of the number of non-repeating multi-word groups to the total number of multi-word groups and the ratio of the number of non-repeating words to the total number of words to obtain the difference degree.

8. A text generation apparatus, characterized in that, The device includes: An acquisition module is used to acquire a set of word order sequences, including multiple word order sequences, generated based on a reference text, and a predicted value of the word order sequences; wherein the predicted value is used to represent the generation probability of the word order sequence; wherein the set of word order sequences is divided into a first word order sequence group and a second word order sequence group; wherein the word order sequences included in the first word order sequence group are designated as the first word order sequence, and the word order sequences included in the second word order sequence group are designated as the second word order sequence; The calculation module is used to calculate the difference between the first word sequence in the first word sequence group and the second word sequence in the second word sequence group; wherein the difference represents the degree of distinction between the first word sequence and the second word sequence; An adjustment module is used to adjust the predicted value of the first word sequence in the first word sequence group based on the difference degree. The partitioning module is used to partition at least a portion of the first word sequence in the first word sequence group into the second word sequence group based on the predicted value of the first word sequence in the adjusted first word sequence group, so as to serve as the second word sequence in the second word sequence group; A generation module is used to generate extended text including corresponding second word sequences based on the second word sequence group; wherein the extended text has the same intent as the base text.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 7.

Images