Text processing method, device, equipment and storage medium
By combining rule-based sentence transformation and machine learning model-based text retelling, the problem that traditional retelling methods cannot guarantee diversity and quality at the same time is solved, and the dual improvement of diversity and quality of text retelling is achieved, which is suitable for corpus expansion and other Natural language processing tasks.
Patent Information
- Application Number
- CN202010437018.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-05-21
- Publication Date
- 2024-11-12
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional text retelling methods cannot guarantee the diversity and generation quality of retelling text at the same time. Rule-based retelling methods have poor diversity, while machine learning-based retelling methods have unstable generation effects.
Combining rule-based sentence transformation and machine learning model-based text retelling, we generate target text as retelling of source text, and complement each other through sentence transformation and machine learning generation models to improve the diversity and quality of retelling text.
The dual guarantee of diversity and quality of text retelling is achieved, making the retelling scheme more complete and robust, and can be used in corpus expansion and other natural language processing tasks to improve the accuracy and efficiency of the model.
Smart Images

Figure CN111814451B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure generally relates to the field of computers, and more particularly to a text processing method, apparatus, device, and computer-readable storage medium. Background Art
[0002] Text generation is an important branch of natural language processing (NLP) technology, which can automatically generate natural language text based on input data. Text generation is an advanced stage of natural language understanding and an important means to achieve intelligent text processing. Text generation technology can be applied to machine translation, dialogue systems, text summarization, article writing, text retelling, image annotation, etc.
[0003] Paraphrase is a type of text generation technology that can express the source text in one or more different ways without changing the semantics of the source text, or the ideas expressed are basically consistent with the source text. It is also called "rewriting". Paraphrase technology is widely used. For example, in machine translation systems, paraphrase technology can be used to simplify complex input texts. In information retrieval systems, paraphrase technology can be used to rewrite user queries, and so on. Summary of the invention
[0004] According to some embodiments of the present disclosure, a text processing method, apparatus, device, and computer-readable storage medium are provided.
[0005] In a first aspect of the present disclosure, a text processing method is provided. The method includes generating a first set of paraphrased texts corresponding to a source text by performing sentence conversion on a source text, wherein the source text includes sentences or parts of sentences. The method also includes generating a second set of paraphrased texts corresponding to the source text by a machine learning generation model. In addition, the method also includes obtaining a set of target texts corresponding to the source text based on at least the first set of paraphrased texts and the second set of paraphrased texts.
[0006] In a second aspect of the present disclosure, a text processing device is provided. The device includes a first text generation module, a second text generation module, and a target text acquisition module. The first text generation module is configured to generate a first group of paraphrased texts corresponding to a source text by performing sentence conversion on a source text, wherein the source text includes sentences or parts of sentences. The second text generation module is configured to generate a second group of paraphrased texts corresponding to the source text through a machine learning generation model. The target text acquisition module is configured to obtain a group of target texts corresponding to the source text based on at least the first group of paraphrased texts and the second group of paraphrased texts.
[0007] In a third aspect of the present disclosure, an electronic device is provided, comprising a memory and a processor, wherein the memory is used to store computer executable instructions, and the computer executable instructions are executed by the processor to implement the method according to the first aspect of the present disclosure.
[0008] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method according to the first aspect and of the present disclosure.
[0009] In the embodiment of the present disclosure, rule-based sentence conversion and machine learning model-based text paraphrase are used simultaneously to generate a target text as a paraphrase of the source text. Therefore, according to the embodiment of the present disclosure, not only the diversity of the text paraphrase can be guaranteed, but also the quality of the text paraphrase can be guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] The features, advantages and other aspects of various implementations of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Several implementations of the present disclosure are shown here in an exemplary and non-limiting manner. In the accompanying drawings:
[0011] Figure 1 An example environment of corpus expansion according to an embodiment of the present disclosure is shown;
[0012] Figure 2 A flowchart of a text processing method according to an embodiment of the present disclosure is shown;
[0013] Figure 3 A schematic diagram showing the architecture of a text retelling system according to an embodiment of the present disclosure;
[0014] Figure 4 A flowchart of a method for retrieving similar sentences according to an embodiment of the present disclosure is shown;
[0015] Figure 5A A schematic diagram showing a sentence conversion process according to an embodiment of the present disclosure is shown;
[0016] Figure 5B A matching diagram of sentence conversion rules according to an embodiment of the present disclosure is shown;
[0017] Figure 5C A schematic diagram showing synonym replacement according to an embodiment of the present disclosure is shown;
[0018] Figure 6 A schematic diagram showing the architecture of a machine learning generation model according to an embodiment of the present disclosure;
[0019] Figure 7A schematic diagram showing a method for training a robot using a text retelling method according to an embodiment of the present disclosure is shown;
[0020] Figure 8 A block diagram showing a text processing apparatus according to an embodiment of the present disclosure; and
[0021] Fig. 9 A block diagram of a computing device is shown in which one or more embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION
[0022] The preferred implementations of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the preferred implementations of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the implementations set forth herein. On the contrary, these implementations are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.
[0023] As used herein, the term "including" and its variations mean open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "based at least in part on". The terms "embodiment" and "some embodiments" mean "at least some embodiments". The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.
[0024] The purpose of text paraphrasing is to express a sentence in a different way without changing the semantics. Traditional text paraphrasing mainly includes rule-based paraphrasing methods or machine learning-based paraphrasing methods. Rule-based paraphrasing methods use constructed rules to generate new sentences. They have the advantages of cold start and controllability. However, due to the limited coverage of rules, this method cannot be generalized and has poor diversity. Machine learning-based paraphrasing methods learn the rules of text paraphrasing construction from a large number of samples. However, the quality of the paraphrased text generated by this method is unstable and the generation effect is poor. Therefore, traditional text paraphrasing methods cannot guarantee both the diversity and generation quality of text paraphrasing.
[0025] To this end, the embodiments of the present disclosure propose a hybrid text paraphrase scheme that combines a rule-based paraphrase method and a machine learning-based paraphrase method. In the embodiments of the present disclosure, rule-based sentence conversion and machine learning model-based text paraphrase are used simultaneously to generate a target text as a paraphrase of the source text. Therefore, according to the embodiments of the present disclosure, not only the diversity of the text paraphrase can be guaranteed, but also the quality of the text paraphrase can be guaranteed, making the text paraphrase scheme more complete and robust. In addition, some embodiments of the present disclosure also propose an improved machine learning generation model. The following will refer to the attached Figure 1-9 Some example embodiments of the present disclosure are described.
[0026] Figure 1 An example environment 100 of corpus expansion according to an embodiment of the present disclosure is shown. The text retelling scheme of the embodiment of the present disclosure can generate one or more target texts of the source text, and thus can be used for corpus expansion, which refers to generating more corpora based on existing corpora, thereby improving the model training effect. Corpus generation generates more similar corpora based on a batch of original corpora to enrich the corpus, so as to improve the accuracy of tasks such as text classification and dialogue intent recognition, and significantly reduce the manpower and time cost of users manually writing similar corpora.
[0027] like Figure 1 As shown in the environment 100, the original corpus 110 includes some corpora that have been annotated and / or manually organized, such as the original sentence 111 "What good movies are recommended" (also referred to as "source text"). According to an embodiment of the present disclosure, the corpus in the original corpus 110 can be expanded to obtain an expanded corpus 120 with more corpora by textually retelling the corpus. Figure 1 As shown, the original sentence 111 generates multiple paraphrases 121 (also referred to as "target text") by expansion. In addition to the original sentence 111, the multiple paraphrases 121 also include multiple paraphrases with the same semantics as the original sentence 111 but different expressions. According to an embodiment of the present disclosure, the multiple paraphrases 121 include both paraphrases generated based on rules, such as a paraphrase with a changed sentence structure "What good movies can you recommend?" and paraphrases generated by a machine learning model, such as a machine-generated paraphrase "What good movies can you recommend recently?" In this way, the generated paraphrases are not only more diverse, but also of better quality. Therefore, the automatically expanded corpus of the embodiment of the present disclosure can achieve better diversity and quality.
[0028] In some embodiments, the method of the embodiment of the present disclosure can expand the corpus for the robot platform, including single sentence expansion and batch expansion. Single sentence expansion is based on the expansion of a single corpus of an existing single category, while batch expansion is based on the simultaneous generation of corpora of multiple categories in a certain field. The characteristic of batch expansion is that all corpora are used as input to generate sentences that conform to the distribution of the existing sample space instead of using single sentences for repetition. For example, the corpus generated under the catering corpus is corpus related to the catering field, and it is impossible to generate corpus in other fields. It should be understood that although in Figure 1 In the text retelling example, corpus expansion is used as an example application scenario. The text retelling method disclosed in the present invention can also be applied to other scenarios, such as text enhancement. The text enhancement function can be used as a general function to provide rich text for common natural language processing tasks, such as text security recognition, customer service text classification, etc., thereby improving the accuracy of the model.
[0029] Figure 2 A flowchart of a text processing method 200 according to an embodiment of the present disclosure is shown, and the method 200 can be executed by a single computing device or multiple computing devices. In box 202, a first group of paraphrased texts corresponding to a source text is generated by performing sentence conversion on a source text, wherein the source text is one or more sentences, or can also be a part of a sentence. For example, the source text can be a seed sentence of a text paraphrase, such as an original sentence and / or a related sentence retrieved based on the original sentence, and the original sentence is subjected to sentence conversion and / or synonym replacement based on a pre-established rule, so as to generate a group of paraphrased sentences including at least one paraphrased sentence according to the rule.
[0030] In block 204, a second set of paraphrased texts corresponding to the source text is generated by a machine learning generation model. For example, a machine learning generation model (such as a deep learning model) is trained using training sentence pairs, and then the original sentences are input into the trained machine learning generation model to generate another set of paraphrased sentences including at least one paraphrase sentence.
[0031] In box 206, a group of target texts corresponding to the source texts is obtained based on at least the first group of paraphrased texts and the second group of paraphrased texts. For example, after obtaining a group of paraphrased sentences generated according to the rules and another group of paraphrased sentences generated according to the machine learning model, the two groups of paraphrased sentences can be combined to form a final group of target texts. Alternatively, the two groups of paraphrased sentences can be used as each other's input texts, and the respective texts are generated again, and this process is repeated until the stop condition is met. Alternatively or additionally, after obtaining several groups of paraphrased sentences, all the paraphrased sentences can also be sorted and screened, so as to select one or more paraphrased sentences with better quality as the final target texts.
[0032] Therefore, according to the method 200 of various embodiments of the present disclosure, rule-based sentence conversion and model generation based on a machine learning model are used simultaneously to generate a target text as a paraphrase of a source text. Therefore, the paraphrase text generated according to the embodiment of the present disclosure can not only ensure the diversity of the paraphrase text, but also ensure the quality of text generation.
[0033] Figure 3 Schematic diagram of the architecture of a text paraphrase system 300 according to an embodiment of the present disclosure is shown. The text paraphrase system 300 can output one or more paraphrase sentences 350 based on an input original sentence 310. Figure 3 As shown, the text retelling system 300 includes a retrieval module 320, a rule-based generation module 331, a machine learning generation model 332, and a sorting and screening module 340. Figure 4 An example retrieval process of the retrieval module 320 is described in reference to Figures 5A-5CAn example sentence conversion process of the rule-based generation module 331 is described, and reference is made to Figure 6 An example architecture for a machine learning generative model 332 is described.
[0034] refer to Figure 3 For the original sentence 310 to be paraphrased, the retrieval module 320 first retrieves relevant sentences in a pre-built text library, sorts the relevant sentences returned by the retrieval, and uses the qualified sentences as seed sentences. The seed sentences are then respectively input into the rule-based generation module 331 and the machine learning generation model 332 for paraphrase sentence generation. The rule-based generation module 331 generates paraphrase sentences based on pre-established rules, which mainly involve syntactic rules and / or word-level expansion, etc. The machine learning generation model 332 is a trained model that can be used to generate paraphrase sentences. In the embodiment of the present disclosure, the machine learning generation model 332 is an improved supervised variational autoencoder. In some embodiments, the paraphrase sentences generated by the rule-based generation module 331 and the machine learning generation model 332 can be used as each other's input to generate paraphrase sentences again. This process can continue for a predetermined number of rounds, or continue until no more paraphrase sentences can be generated. In this way, more paraphrase sentences can be obtained. Finally, the paraphrases generated by the rule-based generation module 331 and the machine learning generation model 332 are input into the sorting and screening module 340 , and the paraphrases 350 that meet the conditions are finally output.
[0035] For each paraphrase generated by the rule-based generation module 331 and the machine learning generation model 332, the sorting and screening module 340 can calculate its language perplexity score and / or semantic similarity score, and rank all paraphrases based on the score, and use one or more paraphrases with higher scores as the final output result. For example, a language model can be trained on an encyclopedia corpus, and the language perplexity of each paraphrase can be scored by the language model. The lower the perplexity, the smoother the sentence. For another example, the bidirectional encoder representation (BERT) model from the transformer can be used to encode the sentence vector of each paraphrase and the original sentence 310, and the cosine similarity is used to calculate the semantic similarity between them. The higher the semantic similarity score of the two sentences, the more similar they are in semantics. It should be understood that any existing or future developed semantic similarity calculation method or tool can be used in combination with the embodiments of the present disclosure to determine the semantic similarity between two sentences or two paragraphs of text. In this way, the paraphrase sentence finally generated can be made more authentic and retain the semantics of the original sentence.
[0036] Reference now Figure 4, which shows a flow chart of a method 400 for the retrieval module 320 to retrieve similar sentences. In box 402, the retrieval module 320 obtains the original sentence 310 to be repeated. In box 404, the retrieval module 320 retrieves sentences related to the original sentence in the database. For example, a question text library can be collected from a question-answering community on the Internet and an index can be established. The original sentence 310 can be a question sentence, and then the question sentence related to the original sentence 310 can be retrieved in the question text library through a full-text search engine (ES), and the retrieval results are sorted to obtain a group of question sentences.
[0037] In box 406, the retrieval module 320 screens the group of questions according to semantic similarity to generate seed sentences. For example, based on the semantic similarity between the original sentence 310 and each question in the group of questions, one or more questions with higher scores can be selected from the group of questions as seed sentences for paraphrasing. In some embodiments, the group of questions can also be filtered, for example, to remove sentences with large length differences and complex sentences. Next, the seed sentences obtained by the retrieval module 320 are input to the rule-based generation module 331 and the machine learning generation model 332 for generating paraphrasing sentences.
[0038] Figure 5A A schematic diagram of a sentence conversion process 500 performed by a rule-based generation module 331 according to an embodiment of the present disclosure is shown. The sentence conversion process 500 mainly involves two generation methods: lexical rule generation and vocabulary expansion. In box 502, a seed sentence is obtained from the retrieval module 320, or the original sentence 310 is directly obtained as a seed sentence, for example, the seed sentence 503 "how to register a chat account". In box 504, the seed sentence is segmented and tagged with parts of speech. For example, "how" in the seed sentence 503 is automatically tagged as a pronoun r, "register" is automatically tagged as a verb v, and "chat account" is automatically tagged as a noun phrase np, as shown in 505.
[0039] In box 506, the annotated results are converted into sentences. Based on the part-of-speech tagging results in 505, the sentence structure of the seed sentence 503 can be obtained as "how [v][np]"; then, according to the conversion rule 507, it can be converted into the sentence structure "[np] how [v]". A rule matching library or rule object library including multiple sentence conversion rules can be generated. For example, an object-level regular expression can be constructed. The traditional regular expression method is based on the format matching of the string, and the object level adds words and their corresponding parts of speech. The regular expression at the object level performs regular matching on these two types of information respectively. In some embodiments, a matching graph can be constructed for all conversion rules, and the matching graph is used to complete the conversion relationship between user-defined sentences.
[0040] Figure 5B A matching graph of sentence conversion rules is shown. Figure 5BThe matching graph includes 5 nodes, nodes 531, 532, 533, 534, and 535, each of which represents a sentence. The directed edges between the nodes represent the conversion relationship between these sentence patterns. For example, edge 542 indicates that the sentence patterns of node 531 and node 532 can be converted to each other. And edge 541 indicates that the sentence pattern "how [v][np]" of node 531 can be converted to the sentence pattern "[np] how [v]" of node 533. The reverse conversion is not necessarily possible. For example, the sentence "I want to know how to register" cannot be converted to "I want to know how to register." In addition, according to Figure 5B The matching graph in can also obtain the following sentence conversion rules:
[0041] How to [v] [np], [v] [np] related consultation => [np] how to [v], [np] how to [v]
[0042] [np]How to [v]=>[np]How to [v]
[0043] Alternatively or additionally, phrase-level paraphrase may be performed, and rules and templates for phrase paraphrase may be generated. For example, the conversion between "in my opinion" and "I feel" belongs to the phrase paraphrase template.
[0044] return Figure 5A In block 508, a complete sentence is generated and at least one word in the sentence is replaced with a synonym. For example, according to the sentence conversion rule shown in 507, the seed sentence 503 is converted into "how to register a chat account", and in addition, "account" can be replaced with a synonym "account" to generate a sentence "how to register a chat account", as shown in 509.
[0045] In some embodiments, synonym replacement may be performed by calculating semantic similarity between words. Figure 5C A schematic diagram of synonym replacement according to an embodiment of the present disclosure is shown. Synonym replacement is a simple vocabulary-based text enhancement method. When the sample is limited, synonym replacement can improve the accuracy of classification. For example, stop words in a sentence can be ignored, one or more words can be randomly selected from the sentence, and then synonyms can be randomly selected from the synonym dictionary and replaced. Figure 5C As shown, synonyms have a score to indicate how close they are to the original word (e.g. Figure 5CIn some embodiments, the similarity of all synonyms can be calculated (the original word similarity is counted as 1), and the similarity of the generated sentence can be obtained by multiplication. Several results with the largest similarity can be taken as synonym replacement results. In addition, the score value between synonyms can be the cosine similarity between word vectors. When calculating the synonyms of a word, all words can be traversed, and the word with the highest score with the original word is taken as a synonym. In order to optimize the retrieval efficiency, the word vector table can be used to construct a K-dimensional (KD) tree to improve the retrieval efficiency.
[0046] Figure 6 FIG. 3 is a schematic diagram showing an example architecture of a machine learning generation model 332 according to an embodiment of the present disclosure. Figure 6 As shown, the machine learning generation model 332 is an improved variational autoencoder including an encoder 610 and a decoder 630. The variational autoencoder is a deep learning model. Different from the traditional variational autoencoder, the architecture of the machine learning generation model 332 of the embodiment of the present disclosure is used to generate a paraphrase of the original sentence, that is, the input of the machine learning generation model 332 also includes the original sentence. In addition, the machine learning generation model 332 of the embodiment of the present disclosure also introduces a copy mechanism, so that the problem of unregistered words can be solved.
[0047] exist Figure 6 In the architecture, a pair of original sentence 601 and paraphrase sentence 602 is used as the input of encoder 610, the original sentence 601 is used as the input of decoder 630, and the paraphrase sentence 602 is used as the output of decoder 630 to train the encoder 610 and decoder 630 in the variational autoencoder, so that the predicted output of decoder 630 is the same as the true output and the output of encoder 610 conforms to the standard normal distribution.
[0048] The encoder 610 includes an original sentence encoder 611 of a long short-term memory network (LSTM) and a retelling sentence encoder 612 of the LSTM, wherein the original sentence encoder 611 is used to encode each word 601 in the original sentence and transmit the encoding to the retelling sentence encoder 612. The retelling sentence encoder 612 is used to encode each word 602 in the retelling sentence. The output of the encoder 610 is a mean 621 and a variance 622. The distribution of the output of the encoder 610 conforms to the standard normal distribution through model training.
[0049] The decoder 630 includes an LSTM original sentence encoder 631 and an LSTM paraphrase decoder 632. A vector of a predetermined dimension, such as a latent variable 623, is randomly sampled from a standard normal distribution and input into the paraphrase decoder 632. By random sampling, the diversity of the paraphrases generated by the machine learning generation model 332 can be better. The original sentence encoder 631 is used to encode each word 601 in the original sentence, and generate an attention distribution 633, thereby obtaining a context vector 634. The encoding vector of the original sentence word is also sent to the paraphrase decoder 632, which generates a vocabulary distribution 635 based on the encoding vector and latent vector 623 of the original sentence, and calculates the generation probability 636 of generating words from the vocabulary in combination with the context vector 634, thereby determining the copy probability of copying words from the original sentence (the sum of the generation probability and the copy probability is 1). Finally, by multiplying the vocabulary distribution 635 by the generation probability in box 637, and multiplying the attention distribution 633 by the copy probability in box 638, the final distribution 639, i.e., the predicted output of the model, can be obtained. Therefore, by introducing the copy mechanism in the pointer generation network in the decoder 630, the weight distribution in the attention can be used as a probability guide to copy the words in the input as output, thereby solving the problem of out-of-vocabulary (OOV) words that do not exist in the vocabulary of the training corpus.
[0050] In the training phase of the machine learning generation model 332, the loss function of the machine learning generation model 332 training may include two parts, the first part is the reconstruction error, that is, the loss between the model's predicted output (i.e., generated sentence) and the actual output (real paraphrase sentence), and the second part is the Kullback-Leibler (KL) divergence, so that the output of the encoder 610 tends to the standard normal distribution Z~N(0,1), thereby ensuring that the model has sufficient generation capability. The parameters of the four LSTMs in the original sentence encoder 611, the paraphrase encoder 612, the original sentence encoder 631, and the paraphrase encoder 632 are not shared.
[0051] In the use phase of the machine learning generation model 332, a hidden variable z is first randomly sampled from the standard normal distribution, and then the hidden variable z is spliced with each word vector in the paraphrase decoder 632. The original sentence encoder 631 encodes the original sentence and passes the code to the paraphrase decoder 632. The paraphrase decoder 632 decodes step by step, and each step forms a vector of the size of the vocabulary, and is converted into a probability by the softmax function, and finally generates the corresponding paraphrase. At the time of output, several words with the highest probability can be selected as candidates, and the prediction is used as the input of the next word. In this process, beam search can be used to improve efficiency until the generation of sentences ends. Therefore, the machine learning generation model 332 proposed in the embodiment of the present disclosure can generate more diverse paraphrases, and can also solve the problem of OOV words. In this way, the paraphrase generated by the machine learning generation model 332 according to the embodiment of the present disclosure can not only have OOV words, but also have better diversity.
[0052] Therefore, the text paraphrase system 300 of the embodiment of the present disclosure adopts a hybrid text paraphrase method based on rules and machine learning models, and sorts and filters based on language fluency and / or semantic similarity, thereby providing users with better paraphrase generation results. In addition, the paraphrase sentences generated according to the embodiment of the present disclosure can be used to enrich the training corpus, thereby improving the accuracy of systems such as intelligent dialogue interaction systems.
[0053] Figure 7 FIG. 2 shows a schematic diagram of using the text retelling method of an embodiment of the present disclosure to train a robot. Figure 7 As shown, the corpus includes a question-answer pair 710, which includes a question 711 and a corresponding answer 712. In order to expand the corpus to improve the model training effect, the text paraphrase method of the embodiment of the present disclosure can be used to generate multiple paraphrases of question 711, such as questions 713, 714, and 715, wherein questions 713, 714, and 715 have the same semantics as question 711 but are expressed in different forms. Then, based on the generated questions 713, 714, and 715, they can be combined with the answer 712 respectively to generate four question-answer pairs 720. Next, these four question-answer pairs 720 are used to train the robot model in box 730. Since the robot model uses more question-answer pairs for training, the trained robot model has better effect.
[0054] See above for Figures 1 to 7 An example of a method according to the present disclosure is described in detail, and implementations of corresponding devices and apparatuses will be described hereinafter.
[0055] According to an exemplary implementation of the present disclosure, a text processing device 800 is provided, and the text processing device 800 includes a first text generation module 810, a second text generation module 820, and a target text acquisition module 830. The first text generation module 810 is configured to generate a first group of paraphrased texts corresponding to the source text by performing sentence conversion on the source text, wherein the source text includes sentences or parts of sentences. The second text generation module 820 is configured to generate a second group of paraphrased texts corresponding to the source text through a machine learning generation model. The target text acquisition module 830 is configured to obtain a group of target texts corresponding to the source text based on at least the first group of paraphrased texts and the second group of paraphrased texts.
[0056] In some embodiments, the target text obtaining module 830 includes: a third text generating module, configured to generate a third group of paraphrase texts corresponding to the second group of paraphrase texts by performing sentence conversion on the second group of paraphrase texts; a fourth text generating module, configured to generate a fourth group of paraphrase texts corresponding to the first group of paraphrase texts through a machine learning generation model; and a target text determining module, configured to determine a group of target texts corresponding to the source texts based on the first group of paraphrase texts, the second group of paraphrase texts, the third group of paraphrase texts and the fourth group of paraphrase texts.
[0057] In some embodiments, the target text determination module includes: a combination module configured to obtain a plurality of paraphrase texts by combining a first group of paraphrase texts, a second group of paraphrase texts, a third group of paraphrase texts, and a fourth group of paraphrase texts; a score determination module configured to determine a score of at least one of the following items of a paraphrase text in the plurality of paraphrase texts: a language perplexity indicating a degree of fluency of the paraphrase text, and a semantic similarity between the paraphrase text and a source text; and a target text selection module configured to select a group of target texts from the plurality of paraphrase texts based on the determined scores.
[0058] In some embodiments, the source text is a question, and the apparatus 800 further includes: a retrieval module configured to retrieve a group of questions related to the source text in a question text library; and a question selection module configured to select one or more questions from the group of questions based on the semantic similarity between the source text and each question in the group of questions.
[0059] In some embodiments, the first text generation module 810 includes: a part-of-speech analysis module, configured to perform word segmentation and part-of-speech tagging on the source text; a sentence conversion module, configured to perform sentence conversion on the annotated source text; and a synonym replacement module, configured to perform synonym replacement on at least one word in the converted source text.
[0060] In some embodiments, the machine learning generation model is a variational autoencoder comprising an encoder and a decoder, and the device 800 further comprises: a training module configured to train the variational autoencoder using a pair of original sentence and paraphrase sentence as input of the encoder, the original sentence as input of the decoder, and the paraphrase sentence as output of the decoder, so that the output of the encoder conforms to a standard normal distribution.
[0061] In some embodiments, the second text generation module 820 includes: a random sampling module configured to randomly sample a predetermined dimensional vector from a standard normal distribution; and a decoding module configured to generate a paraphrase text corresponding to the source text through a decoder based on the source text and the predetermined dimensional vector.
[0062] In some embodiments, the decoding module includes: a generation probability determination module, configured to determine the generation probability of generating a word from a vocabulary through a pointer generation network in a decoder, the pointer generation network being used to implement the copying of words; a copy probability determination module, configured to determine the copy probability of copying words from a source text based on the generation probability; and a paraphrase text generation module, configured to generate a paraphrase text based on the generation probability and the copy probability.
[0063] In some embodiments, the apparatus 800 further includes: a corpus expansion module configured to expand the training corpus through the obtained set of target texts; and a target model training module configured to train the target model using the expanded training corpus.
[0064] In some embodiments, the source text is a question in a question-answer pair, and the device 800 also includes: a question-answer pair generation module, configured to generate a plurality of question-answer pairs based on a set of target texts and the answers in the question-answer pairs; and a robot training module, configured to train a robot model using the generated plurality of question-answer pairs.
[0065] Therefore, some embodiments of the present disclosure propose a hybrid integrated text retelling system based on rules and machine learning models, and combine the sorting mechanism of language fluency and text similarity to improve the quality of text generation. It can expand the user's domain corpus to improve the accuracy of the target task, thereby improving the user's corpus organization experience.
[0066] Fig. 9 A block diagram illustrating a computing device / server 900 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that Fig. 9 The computing device / server 900 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein.
[0067] like Fig. 9As shown, computing device / server 900 is in the form of a general computing device. Components of computing device / server 900 may include, but are not limited to, one or more processors or processing units 910, memory 920, storage device 930, one or more communication units 940, one or more input devices 950, and one or more output devices 960. Processing unit 910 may be an actual or virtual processor and is capable of performing various processes according to a program stored in memory 920. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to increase the parallel processing capabilities of computing device / server 900.
[0068] The computing device / server 900 typically includes a plurality of computer storage media. Such media may be any available media accessible to the computing device / server 900, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 920 may be a volatile memory (e.g., registers, caches, random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 930 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data (e.g., training data for training) and may be accessed within the computing device / server 900.
[0069] The computing device / server 900 may further include additional removable / non-removable, volatile / non-volatile storage media. Fig. 9 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. Memory 920 may include a computer program product 925 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.
[0070] The communication unit 940 enables communication with other computing devices via a communication medium. Additionally, the functions of the components of the computing device / server 900 can be implemented in a single computing cluster or multiple computing machines that can communicate via a communication connection. Therefore, the computing device / server 900 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.
[0071] Input device 950 may be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 960 may be one or more output devices, such as a display, speaker, printer, etc. Computing device / server 900 may also communicate with one or more external devices (not shown) as needed, such as storage devices, display devices, etc., through communication unit 940, with one or more devices that allow users to interact with computing device / server 900, or with any device that allows computing device / server 900 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).
[0072] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.
[0073] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.
[0074] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable information presentation device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable information presentation device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable information presentation device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0075] Computer-readable program instructions may be loaded onto a computer, other programmable information presentation apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable information presentation apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable information presentation apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0076] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.
[0077] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.
Claims
1. A text processing method, comprising: Generating a first set of paraphrased texts corresponding to the source text by performing sentence transformation on the source text, wherein the source text includes sentences or a part of sentences; Generate a second set of paraphrased texts corresponding to the source texts through a machine learning generative model; as well as Based at least on the first set of paraphrased texts and the second set of paraphrased texts, obtaining a set of target texts corresponding to the source texts; Wherein, obtaining a set of target texts corresponding to the source texts comprises: Generating a third group of paraphrase texts corresponding to the second group of paraphrase texts by performing sentence conversion on the second group of paraphrase texts; Generate a fourth set of paraphrased texts corresponding to the first set of paraphrased texts by the machine learning generation model; and The set of target texts corresponding to the source texts is determined based on the first set of paraphrase texts, the second set of paraphrase texts, the third set of paraphrase texts, and the fourth set of paraphrase texts.
2. The method of claim 1, wherein determining the set of target texts corresponding to the source texts comprises: Obtaining a plurality of paraphrase texts by combining the first group of paraphrase texts, the second group of paraphrase texts, the third group of paraphrase texts, and the fourth group of paraphrase texts; determining a score of at least one of the following for a paraphrase text among the plurality of paraphrases texts: a language perplexity indicating a fluency of the paraphrase text, and a semantic similarity between the paraphrase text and the source text; and The set of target texts is selected from the plurality of paraphrase texts based on the determined scores.
3. The method according to claim 1, wherein the source text is a question sentence, and the method further comprises: Retrieving a set of question sentences related to the source text in a question text library; as well as One or more questions are selected from the set of questions based on the semantic similarity between the source text and each question in the set of questions.
4. The method of claim 1, wherein generating a first set of paraphrased texts corresponding to the source text comprises: Performing word segmentation and part-of-speech tagging on the source text; Perform sentence transformation on the annotated source text; as well as Perform synonym replacement on at least one word in the converted source text.
5. The method of claim 1, wherein the machine learning generative model is a variational autoencoder comprising an encoder and a decoder, and the method further comprises: The variational autoencoder is trained by taking a pair of original sentence and paraphrase sentence as input of the encoder, taking the original sentence as input of the decoder and taking the paraphrase sentence as output of the decoder, so that the output of the encoder conforms to the standard normal distribution.
6. The method according to claim 5, wherein generating a second set of paraphrased texts corresponding to the source texts by using a machine learning generation model comprises: Randomly sampling a vector of predetermined dimension from the standard normal distribution; as well as The decoder generates a paraphrase text corresponding to the source text based on the source text and the predetermined dimensional vector.
7. The method of claim 6, wherein generating, by the decoder, a paraphrase text corresponding to the source text comprises: Determining the generation probability of a word from a vocabulary by a pointer generation network in the decoder, wherein the pointer generation network is used to implement the copying of the word; Determining a copy probability of copying a word from the source text based on the generation probability; as well as The paraphrase text is generated according to the generation probability and the copy probability.
8. The method according to any one of claims 1 to 7, further comprising: Expand the training corpus by obtaining a set of target texts; as well as Use the expanded training corpus to train the target model.
9. The method according to any one of claims 1 to 7, wherein the source text is a question in a question-answer pair, and the method further comprises: Generate a plurality of question-answer pairs based on the set of target texts and the answers in the question-answer pairs; as well as Use the generated multiple question-answer pairs to train the robot model.
10. A text processing device, comprising: A first text generation module is configured to generate a first set of paraphrase texts corresponding to a source text by performing sentence conversion on the source text, wherein the source text includes a sentence or a part of a sentence; A second text generation module is configured to generate a second set of paraphrased texts corresponding to the source text by using a machine learning generation model; as well as a target text obtaining module, configured to obtain a set of target texts corresponding to the source texts based at least on the first set of paraphrased texts and the second set of paraphrased texts; Wherein, the target text obtaining module is further configured as follows: Generating a third group of paraphrase texts corresponding to the second group of paraphrase texts by performing sentence conversion on the second group of paraphrase texts; Generate a fourth group of paraphrased texts corresponding to the first group of paraphrased texts by using the machine learning generation model; as well as The set of target texts corresponding to the source texts is determined based on the first set of paraphrase texts, the second set of paraphrase texts, the third set of paraphrase texts, and the fourth set of paraphrase texts.
11. An electronic device, comprising: processor; as well as A memory storing computer executable instructions, wherein the computer executable instructions, when executed by the processor, are configured to implement the method according to any one of claims 1 to 9.
12. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions are executed by a processor to implement the method according to any one of claims 1 to 9.
