Automatic generation method, device, equipment and storage medium for paraphrase text
By generating and matching dependency paths, the fluency and adaptability of machine retelling texts are improved, the problem of children's dyslexia is solved, and text that is more suitable for children's logical thinking is provided.
Patent Information
- Application Number
- CN202111440891.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-11-30
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-11-30
AI Technical Summary
The existing machine retelling text is difficult to meet children's simple, easy to understand, vivid and interesting reading requirements, which leads to reading difficulties.
By collecting the training corpus, a first dependency path is generated, a retelling corpus is constructed, a syntax tree of the statement to be repeated is obtained and converted into a dependency tree, and a dependency path in the retelling corpus is matched to generate a smoother retelling text.
It improves the accuracy and generation efficiency of retelling text, adapts to children's logical thinking and understanding ability, and solves the problem of stiff words in existing machines retelling text.
Smart Images

Figure CN114139500B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of natural language processing, and in particular to a method, device, equipment and storage medium for automatically generating a paraphrase text. Background Art
[0002] At present, scripts can be created intelligently through NLG algorithms, sequence generation algorithms, and text analysis technologies. However, scripts created in the above ways have high requirements on children's comprehension ability and do not meet children's requirements for scripts. Children's comprehension ability and concentration are not high. Only texts that are simple, easy to understand, vivid, interesting, and accurate in content are suitable for children to read. Therefore, converting them into scripts that children can understand is the key. Machine translation can convert scripts, but the text is stiff and not fluent, which does not conform to children's thinking logic, making it difficult for children to learn. Summary of the invention
[0003] The present invention provides a method, device, equipment and storage medium for automatically generating a paraphrase text, which can solve the problem that the existing machine paraphrase text cannot well meet the requirements of special groups for reading text due to stiff and unsmooth words.
[0004] In order to solve the above technical problems, a technical solution adopted by the present invention is to provide a method for automatically generating a paraphrase text, comprising:
[0005] Collecting training corpus, generating a first dependency path according to the training corpus, and building a paraphrase corpus based on the first dependency path, wherein the first dependency path is used to describe the dependency relationship between words in a sentence;
[0006] Acquire a sentence to be repeated, preprocess the sentence to be repeated, and generate a syntax tree according to the preprocessed sentence to be repeated;
[0007] Converting the syntax tree into a dependency tree and acquiring a second dependency path corresponding to the sentence to be repeated according to the dependency tree;
[0008] The second dependency path is matched with the first dependency path in the paraphrase corpus, and the training corpus corresponding to the matched first dependency path is used as the paraphrase text of the sentence to be paraphrased.
[0009] According to an embodiment of the present invention, collecting training corpus, generating a first dependency path according to the training corpus, and constructing a paraphrase corpus based on the first dependency path includes:
[0010] Collecting training corpus, extracting key description information from the training corpus and identifying the correctness of the extracted key description information, and using the training corpus with correct key description information as target training corpus;
[0011] Performing word segmentation processing on the target training corpus and performing part-of-speech tagging on the word segmentation processing results;
[0012] Constructing a syntax tree based on the part-of-speech tagging results and converting the syntax tree into a dependency tree;
[0013] The first dependency path is generated according to the dependency tree, and the paraphrase corpus is constructed based on the first dependency path.
[0014] According to an embodiment of the present invention, the acquiring of the sentence to be paraphrased, preprocessing the sentence to be paraphrased, and generating a syntax tree according to the preprocessed sentence to be paraphrased comprises:
[0015] Using a word segmentation algorithm to perform word segmentation processing on the sentence to be repeated to obtain a plurality of ordered word segments;
[0016] Performing part-of-speech tagging on the word segments in sequence;
[0017] Arrange the part-of-speech tagging results in order to generate a syntax tree corresponding to the sentence to be repeated.
[0018] According to an embodiment of the present invention, converting the syntax tree into a dependency tree and acquiring a second dependency path corresponding to the sentence to be repeated according to the dependency tree includes:
[0019] Starting from the leaf nodes of the syntactic tree, each of the word segmentation nodes is attributed to the node of the corresponding word class;
[0020] Obtaining the main word of each part of speech, and attributing the main word in each part of speech to a parent node from bottom to top;
[0021] Obtaining the central word of the sentence to be repeated, and attributing the central word to a root node to generate the dependency tree;
[0022] Starting from the root node of the dependency tree, each of the word segments is encoded in sequence to generate a word sequence, and the second dependency path is generated according to the dependency relationship between each of the word segments in the dependency tree and the word sequence.
[0023] According to an embodiment of the present invention, matching the second dependency path with the first dependency path in the paraphrase corpus and using the training corpus corresponding to the matched first dependency path as the paraphrase text of the sentence to be paraphrased includes:
[0024] Matching the second dependency path with the first dependency path in the paraphrase corpus, and judging whether there is a paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus according to the matching result;
[0025] If there is a paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus, replacing the sentence to be paraphrased with a training corpus corresponding to the first dependency path;
[0026] If there is no paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus, the sentence to be paraphrased is retained.
[0027] According to an embodiment of the present invention, matching the second dependency path with the first dependency path in the paraphrase corpus, and judging whether there is a paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus according to the matching result includes:
[0028] Traversing the first dependency paths in the paraphrase corpus, and calculating the matching degree between the second dependency path and each of the first dependency paths;
[0029] Comparing the matching degree calculation result with a preset value, and determining whether there is a first dependency path matching the second dependency path in the paraphrase corpus according to the comparison result;
[0030] If the matching degree calculation result is greater than or equal to the preset value, it is determined that there is a first dependency path matching the second dependency path in the paraphrase corpus, and it is determined that there is a paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus;
[0031] If the calculated matching result is less than the preset value, it is determined that there is no first dependency path matching the second dependency path in the paraphrase corpus, and it is determined that there is no paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus.
[0032] According to an embodiment of the present invention, traversing the first dependency paths in the paraphrase corpus and calculating the matching degree between the second dependency paths and each of the first dependency paths includes:
[0033] Obtain matching words of the first dependency path and the second dependency path, wherein the matching words are participles having the same dependency relationship and the same glyph;
[0034] The number of matching words and the total number of words in the second dependency path are calculated, and the ratio of the number of matching words to the total number of words is used as the matching degree between the second dependency path and the first dependency path.
[0035] In order to solve the above technical problems, another technical solution adopted by the present invention is to provide an automatic generation device for a paraphrase text, comprising:
[0036] A collection module, used for collecting training corpus, generating a first dependency path according to the training corpus, and building a paraphrase corpus based on the first dependency path, wherein the first dependency path is used for describing the dependency relationship between words in a sentence;
[0037] A generating module, used for acquiring a sentence to be repeated, preprocessing the sentence to be repeated, and generating a syntax tree according to the preprocessed sentence to be repeated;
[0038] A conversion module, configured to convert the syntax tree into a dependency tree and obtain a second dependency path corresponding to the sentence to be repeated according to the dependency tree;
[0039] A matching module is used to match the second dependency path with the first dependency path in the paraphrase corpus, and use the training corpus corresponding to the matched first dependency path as the paraphrase text of the sentence to be paraphrased.
[0040] In order to solve the above technical problems, another technical solution adopted by the present invention is: to provide a computer device, including: a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for automatically generating a paraphrase text when executing the computer program.
[0041] In order to solve the above technical problem, another technical solution adopted by the present invention is: providing a computer storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned method for automatically generating a paraphrase text is implemented.
[0042] The beneficial effects of the present invention are as follows: a first dependency path for describing the dependency relationship between words in a sentence is generated through training corpus, a second dependency path is generated through the sentence to be repeated, and the sentence corresponding to the first dependency path matching the second dependency path is used as the repetition text of the sentence to be repeated, thereby improving the accuracy of repetition and the efficiency of text generation, and at the same time being able to obtain a text that meets the logical thinking and comprehension abilities of special groups of people, thereby solving the problem that the existing machine-repeated text cannot well meet the requirements of special groups for reading texts due to its stiff and unsmooth text. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 is a flow chart of a method for automatically generating a paraphrase text according to an embodiment of the present invention;
[0044] Figure 2 is a flow chart of step S101 in the method for automatically generating a paraphrase text according to an embodiment of the present invention;
[0045] Figure 3 is a schematic diagram of the architecture of a syntax tree and a dependency tree according to an embodiment of the present invention;
[0046] Figure 4 is a flow chart of step S102 in the method for automatically generating a paraphrase text according to an embodiment of the present invention;
[0047] Figure 5 is a flow chart of step S103 in the method for automatically generating a paraphrase text according to an embodiment of the present invention;
[0048] Figure 6 is a schematic diagram of the architecture of a syntax tree and a dependency tree according to another embodiment of the present invention;
[0049] Figure 7 is a flow chart of step S104 in the method for automatically generating a paraphrase text according to an embodiment of the present invention;
[0050] Figure 8 is a schematic diagram of the structure of an automatic generation device for a paraphrase text according to an embodiment of the present invention;
[0051] Fig. 9 is a schematic diagram of the structure of a computer device according to an embodiment of the present invention;
[0052] Fig.10 It is a schematic diagram of the structure of a computer storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION
[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0054] The terms "first", "second" and "third" in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined as "first", "second" and "third" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. All directional indications (such as up, down, left, right, front, back...) in the embodiments of the present invention are only used to explain the relative position relationship, movement, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication also changes accordingly. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.
[0055] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present invention. The appearance of the phrase in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0056] Figure 1 FIG. 1 is a flow chart of a method for automatically generating a paraphrase text according to an embodiment of the present invention. It should be noted that the method of the present invention is not limited to the method of generating a paraphrase text according to an embodiment of the present invention. Figure 1 The process sequence shown is limited. Figure 1 As shown, the method comprises the steps of:
[0057] Step S101: collecting training corpus, generating a first dependency path according to the training corpus, and constructing a paraphrase corpus based on the first dependency path, wherein the first dependency path is used to describe the dependency relationship between words in a sentence.
[0058] In step S101, this embodiment is applied to the generation of children's financial education scripts, and the training corpus can be sentences, paragraphs or texts related to children's financial knowledge. Specifically, the training corpus can include but is not limited to children's books and fairy tale text materials. The training corpus requires to be simple and easy to understand, vivid and interesting, and accurate in content, which can meet the characteristics of children's low comprehension ability and concentration. However, these training corpora may contain incorrect knowledge transmission. Therefore, after collecting the relevant training corpus, it is necessary to use machine automatic screening or manual screening to identify the training corpus, and screen out the training corpus with correct knowledge transmission and upward content guidance. Based on the screened training corpus, the dependency relationship between the words in each sentence in the training corpus is analyzed to generate a first dependency path, and a retelling corpus is constructed based on the first dependency path. In the retelling corpus, a sentence can include multiple words, and each word may have a dependency relationship, so a sentence corresponds to a first dependency path, and a first dependency path can include multiple dependencies. The sentences in the retelling corpus of this embodiment are all used as retellable texts.
[0059] In one possible implementation, see Figure 2 , step S101 further includes the following steps:
[0060] Step S201: collecting training corpus, extracting key description information from the training corpus and identifying the correctness of the extracted key description information, and taking the training corpus with correct key description information as the target training corpus.
[0061] The key description information of this embodiment may be a relevant description of children's financial knowledge. The correctness of the key description information is identified, and the correctly identified key description information is retained to improve the quality of the training corpus.
[0062] Step S202: performing word segmentation processing on the target training corpus and performing part-of-speech tagging on the word segmentation processing results.
[0063] The target training corpus of this embodiment is a sentence related to children's financial knowledge. The target training corpus is segmented by a word segmentation algorithm, and all the words in a sentence are separated by spaces to form multiple word segments. Word segmentation algorithms include forward reverse maximum matching, n-gram, and machine learning. Take the n-gram algorithm as an example to illustrate, segment the sentence, separate all the words in a sentence with spaces, and convert a string into a word sequence. For example, the n-gram algorithm divides each sentence into a sliding window of size N according to bytes, forming a byte segment sequence of length N, each byte segment is called a gram, and the frequency of occurrence of all grams is counted, and filtered according to a pre-set threshold to form a key gram list. Each gram in the list is a feature vector dimension, that is, a word segmentation. For example, "money is the product of commodity exchange" is segmented to obtain the word segmentation "money, is, goods, exchange, product".
[0064] The word segmentation results are tagged based on part-of-speech tagging algorithms (such as HMM and CRF), that is, nouns, verbs, etc. are distinguished and tagged, for example, money (n), is (v), item (n), exchange (v), product (n).
[0065] Step S203: construct a syntax tree based on the part-of-speech tagging result and convert the syntax tree into a dependency tree.
[0066] The syntax tree can describe the relationship between each word, that is, it points out the syntactic collocation relationship between words. The syntax tree generation algorithm is based on the principle of statistical learning. According to a large number of annotated corpora (tree banks), non-terminal symbols, terminal symbols, rule sets and their probability parameters are obtained through machine learning algorithms, and then the dynamic programming algorithm is used to generate the syntax tree for each sentence. If multiple tree structures are encountered during the syntax tree generation process, the one with the highest probability is selected as the best sentence structure.
[0067] The dependency tree can describe the dependency relationship between words. Compared with the syntactic tree, the dependency tree has a simpler structure, fewer levels and nodes, and equivalent conversion can be performed between the syntactic tree and the dependency tree.
[0068] The syntax tree includes multiple nodes used to describe the relationship between words, including root nodes, parent nodes, and leaf nodes. Leaf nodes depend on the parent nodes of the leaf nodes. The root node is the annotation set ROOT of the syntax tree, the parent node is the word class of the word, and the leaf node is a specific word. For example, the syntax tree of "money is the product of commodity exchange" is as follows:
[0069]
[0070] The steps of converting the syntax tree to the dependency tree specifically include: starting from the leaf nodes of the syntax tree, attribute each word node to the node of the corresponding word class; obtain the main word of each word class, and attribute the main word in each word class to the parent node from bottom to top; obtain the central word of the sentence, and attribute the central word to the root node to generate a dependency tree. Figure 3 shown.
[0071] Step S204: Generate a first dependency path according to the dependency tree and construct a paraphrase corpus based on the first dependency path.
[0072] Starting from the root node of the dependency tree, each word is encoded in sequence to generate a word sequence; and the first dependency path is generated according to the dependency relationship between each word in the dependency tree and the word sequence.
[0073] For example, the first dependency path of "money is the product of goods exchange" is:
[0074] Nsubj(yes -2, money -1)
[0075] Root (root-0, is-2)
[0076] Nsubj (yes -2, item -3)
[0077] Nsubj(item -3, exchange -4)
[0078] Ccomp(exchange -4, -5)
[0079] Dobj (of-5, product-6)
[0080] For example, Nsubj (is-2, money-1) represents the relationship between the former and the latter, the former is the parent and the latter is the child, that is, "money" depends on "is", "1" is the code of "money" and "2" is the code of "is".
[0081] Step S102: Acquire the sentence to be paraphrased, preprocess the sentence to be paraphrased, and generate a syntax tree according to the preprocessed sentence to be paraphrased.
[0082] In step S102, the sentence to be repeated is a sentence related to children's financial education that needs to be converted into a sentence that is easy for children to understand. The pre-processing of the sentence to be repeated includes word segmentation processing and part-of-speech tagging processing.
[0083] In one possible implementation, see Figure 4 , step S102 further includes the following steps:
[0084] Step S401: Use a word segmentation algorithm to perform word segmentation processing on the sentence to be repeated to obtain a plurality of ordered word segments.
[0085] The tokenization algorithm is used to split the sentence to be paraphrased, forming multiple tokens. Tokenization algorithms include, for example, forward and backward maximum matching, n-gram, and machine learning. Taking the n-gram algorithm as an example, the sentence to be paraphrased is split, and all the words in the sentence to be paraphrased are separated by spaces. For example, the n-gram algorithm divides the sentence to be paraphrased into byte sequences of size N using a sliding window, forming a sequence of byte segments of length N. Each byte segment is called a gram. The occurrence frequencies of all grams are counted and filtered according to a pre-set threshold to form a list of key grams. Each gram in the list is a dimension of the feature vector, that is, a token. For example, tokenizing the sentence "Money is the product of commodity exchange" gives tokens "Money, is, commodity, exchange, product".
[0086] Step S402: Sequentially perform part-of-speech tagging on the tokens.
[0087] Based on part-of-speech tagging algorithms (such as HMM and CRF), part-of-speech tagging is performed on the tokenization results. That is, nouns, verbs, etc. are distinguished and tagged. For example, money (n), is (v), commodity (n), exchange (v), product (n).
[0088] Step S403: Arrange the part-of-speech tagging results in order to generate a syntactic tree corresponding to the sentence to be paraphrased.
[0089] The syntactic tree can describe the mutual relationship between each token, that is, it indicates the syntactic collocation relationship between tokens. The syntactic tree generation algorithm is based on the principle of statistical learning. According to a large number of annotated corpora (treebanks), through machine learning algorithms, non-terminal symbols, terminal symbols, rule sets, and their probability parameters are obtained, and then the dynamic programming algorithm is used to generate the syntactic tree of each sentence. If multiple tree structures are encountered during the syntactic tree generation process, the one with the highest probability is selected as the best sentence structure.
[0090] The syntactic tree includes multiple nodes used to describe the mutual connection between tokens, specifically including the root node, parent node, and leaf node. The leaf node depends on the parent node of the leaf node. The root node is the annotation set ROOT of the syntactic tree, the parent node is the part-of-speech of the token, and the leaf node is the token. Part-of-speech categories, for example, IP is a simple clause, NP is a noun phrase, VP is a verb phrase, PP is a prepositional phrase, and CP is a phrase indicating a modifying relationship formed by 'de'.
[0091] For example, the syntactic tree of the sentence to be paraphrased "Money is the product of commodity exchange" is as follows:
[0092]
[0093] Step S103: converting the syntax tree into a dependency tree and acquiring a second dependency path corresponding to the sentence to be paraphrased according to the dependency tree.
[0094] In step S103, the dependency tree can describe the dependency relationship between each word segment. Compared with the syntax tree, the dependency tree has a simpler structure, fewer levels and nodes, and the syntax tree and the dependency tree can be equivalently converted. Starting from the root node of the dependency tree, each word segment is encoded in sequence to generate a word sequence; the second dependency path is generated according to the dependency relationship between each word segment in the dependency tree and the word sequence.
[0095] In one possible implementation, see Figure 5 , step S103 further includes the following steps:
[0096] Step S501: starting from the leaf nodes of the syntax tree, attribute each word segmentation node to the node of the corresponding word class;
[0097] Step S502: Obtain the main word of each word class, and attribute the main word in each word class to the parent node from bottom to top;
[0098] Step S503: obtaining the central word of the sentence to be repeated, and attributing the central word to the root node to generate a dependency tree;
[0099] Step S504: Starting from the root node of the dependency tree, each word segment is encoded in sequence to generate a word sequence, and a second dependency path is generated according to the dependency relationship between each word segment in the dependency tree and the word sequence.
[0100] like Figure 6 As shown, for example, the conversion steps between the syntax tree and the dependency tree of the sentence to be repeated "money is the product of commodity exchange" are as follows: attribute "money" to the node NN that controls it, attribute "is" to the node VV that controls it, attribute "commodity" to the node NN that controls it, attribute "exchange" to the node VV that controls it, attribute "of" to the node CC that controls it, and attribute "product" to the node NN that controls it. Attribute "of" in "product of" to its parent node CP, attribute "exchange" to its parent node VP, attribute "commodity" to its parent node NP, attribute the central word "exchange" of the sentence "product of commodity exchange" from node VP to node "IP", attribute "money" to its parent node NP, attribute "is" to its parent node VP, and attribute the central word "is" of "money is" from node VP to node "IP" to obtain a dependency tree.
[0101] The second dependency path of the sentence to be repeated "Money is the product of commodity exchange" is:
[0102] Nsubj(yes -2, money -1)
[0103] Root (root-0, is-2)
[0104] Nsubj(is-2,commodity-3)
[0105] Nsubj(commodity-3, exchange-4)
[0106] Ccomp(exchange -4, -5)
[0107] Dobj (of-5, product-6)
[0108] For example, Nsubj (is-2, money-1) represents the relationship between the former and the latter, the former is the parent and the latter is the child, that is, "money" depends on "is", "1" is the code of "money" and "2" is the code of "is".
[0109] Step S104: Match the second dependency path with the first dependency path in the paraphrase corpus, and use the training corpus corresponding to the matched first dependency path as the paraphrase text of the sentence to be paraphrased.
[0110] In step S104, if the second dependency path of the sentence to be paraphrased is matched with the first dependency path in the paraphrase corpus, and a matching first dependency path can be found, it means that there is a paraphraseable sentence corresponding to the sentence to be paraphrased in the paraphrase corpus, and the training corpus corresponding to the first dependency path is used as the paraphraseable sentence of the sentence to be paraphrased, and the paraphraseable sentence is used to replace the sentence to be paraphrased. If a matching first dependency path is not found in the paraphrase corpus, it means that there is no paraphraseable sentence corresponding to the sentence to be paraphrased in the paraphrase corpus, and the sentence to be paraphrased is kept without being paraphrased.
[0111] In one possible implementation, Figure 7 As shown, step S104 also includes the following steps:
[0112] Step S701: Match the second dependency path with the first dependency path in the paraphrase corpus, and determine whether there is a paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus according to the matching result.
[0113] In step S701, the first dependency paths in the paraphrase corpus are traversed, and the matching degree between the second dependency path and each first dependency path is calculated; the matching degree calculation result is compared with a preset value, and it is determined whether there is a first dependency path matching the second dependency path in the paraphrase corpus according to the comparison result; if the matching degree calculation result is greater than or equal to the preset value, it is determined that there is a first dependency path matching the second dependency path in the paraphrase corpus, and it is determined that there is a paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus; if the matching degree calculation result is less than the preset value, it is determined that there is no first dependency path matching the second dependency path in the paraphrase corpus, and it is determined that there is no paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus.
[0114] In one embodiment, the degree of match between the second dependency path and each first dependency path is calculated as follows: obtain matching words of the first dependency path and the second dependency path; calculate the number of matching words and the total number of words contained in the second dependency path, and use the ratio of the number of matching words to the total number of words as the degree of match between the second dependency path and the first dependency path. The matching words of this embodiment are participles with the same dependency relationship and the same glyph. Specifically, the matching words of the first dependency path and the second dependency path can be matched from two adjacent nodes or from the root node; specifically, words with the same dependency relationship, that is, words with the same encoding, are used as matching targets, and the words with the same encoding are compared to see if they are the same. If the words with the same encoding are the same, they are considered to be matched, and if the words with the same encoding are not the same, they are considered to be mismatched. For example, the sentence to be repeated is "money is the product of commodity exchange", the sentence to be matched is "money is the product of item exchange", the matching word is money, which is the product of exchange, and the matching degree is 83.3%; for another example, the sentence to be repeated is "money is the product of commodity exchange", the sentence to be matched is "money is a currency", the matching word is money, which is two words, and the matching degree is 50%. The preset value can be determined according to the actual situation, which can be 70%. The matching degree of "money is the product of commodity exchange" and "money is the product of item exchange" is 83.3%, which is greater than 70%, and it is determined that there is a repeatable sentence of the sentence to be repeated in the retelling corpus; the matching degree of "money is the product of commodity exchange" and "money is a currency" is 50%, which is less than 70%, and it is determined that there is no repeatable sentence of the sentence to be repeated in the retelling corpus.
[0115] Step S702: If there is a paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus, the sentence to be paraphrased is replaced with the training corpus corresponding to the first dependency path.
[0116] In this embodiment, "money is the product of commodity exchange" is replaced with "money is the product of item exchange", and "money is the product of item exchange" is used as the final text.
[0117] Step S703: If there is no paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus, the sentence to be paraphrased is retained.
[0118] In this embodiment, "Money is the product of commodity exchange" is retained as the final text.
[0119] The automatic generation method of the paraphrase text in the embodiment of the present invention generates a first dependency path for describing the dependency relationship between words in a sentence through training corpus, generates a second dependency path through the sentence to be paraphrased, and uses the sentence corresponding to the first dependency path that matches the second dependency path as the paraphrase text of the sentence to be paraphrased, thereby improving the accuracy of the paraphrase and the efficiency of text generation, and at the same time being able to obtain a text that meets the logical thinking and comprehension abilities of special groups of people, thereby solving the problem that the existing machine-repeated text cannot well meet the requirements of special groups for reading texts due to its stiff and unsmooth text.
[0120] Figure 8 Schematic diagram of the structure of the automatic generation device of the paraphrase text according to the embodiment of the present invention. Figure 8 As shown, the device 80 includes a collection module 81 , a generation module 82 , a conversion module 83 and a matching module 84 .
[0121] The collection module 81 is used to collect training corpus, generate a first dependency path according to the training corpus, and build a paraphrase corpus based on the first dependency path, where the first dependency path is used to describe the dependency relationship between words in a sentence;
[0122] The generating module 82 is used to obtain the sentence to be repeated, pre-process the sentence to be repeated, and generate a syntax tree according to the pre-processed sentence to be repeated;
[0123] The conversion module 83 is used to convert the syntax tree into a dependency tree and obtain a second dependency path corresponding to the sentence to be repeated according to the dependency tree;
[0124] The matching module 84 is used to match the second dependency path with the first dependency path in the paraphrase corpus, and use the training corpus corresponding to the matched first dependency path as the paraphrase text of the sentence to be paraphrased.
[0125] See also Fig. 9 , Fig. 9 FIG. 1 is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. Fig. 9 As shown, the computer device 90 includes a processor 91 and a memory 92 coupled to the processor 91 .
[0126] The memory 92 stores program instructions for implementing the method for automatically generating a paraphrase text described in any of the above embodiments.
[0127] The processor 91 is used to execute the program instructions stored in the memory 92 to automatically generate the paraphrase text.
[0128] The processor 91 may also be referred to as a CPU (Central Processing Unit). The processor 91 may be an integrated circuit chip having the ability to process signals. The processor 91 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0129] See also Fig.10 , Fig.10 Schematic diagram of the structure of the computer storage medium of the embodiment of the present invention. The computer storage medium of the embodiment of the present invention stores a program file 101 that can implement all the above methods, wherein the program file 101 can be stored in the above computer storage medium in the form of a software product, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned computer storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes, or terminal devices such as computers, servers, mobile phones, tablets, etc.
[0130] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0131] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0132] The above are only implementation modes of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A method for automatically generating a paraphrase text, characterized in that: include: Collecting training corpus, generating a first dependency path according to the training corpus, and building a paraphrase corpus based on the first dependency path, wherein the first dependency path is used to describe the dependency relationship between words in a sentence; Acquire a sentence to be repeated, preprocess the sentence to be repeated, and generate a syntax tree according to the preprocessed sentence to be repeated; Converting the syntax tree into a dependency tree and acquiring a second dependency path corresponding to the sentence to be repeated according to the dependency tree; Matching the second dependency path with the first dependency path in the paraphrase corpus, and using the training corpus corresponding to the matched first dependency path as the paraphrase text of the sentence to be paraphrased includes: Matching the second dependency path with the first dependency path in the paraphrase corpus, and judging whether there is a paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus according to the matching result; If there is a paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus, replacing the sentence to be paraphrased with a training corpus corresponding to the first dependency path; If there is no repeatable sentence of the sentence to be repeated in the repeat corpus, retaining the sentence to be repeated; The step of matching the second dependency path with the first dependency path in the paraphrase corpus and judging whether there is a paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus according to the matching result comprises: Traversing the first dependency paths in the paraphrase corpus, and calculating the matching degree between the second dependency path and each of the first dependency paths; Comparing the matching degree calculation result with a preset value, and determining whether there is a first dependency path matching the second dependency path in the paraphrase corpus according to the comparison result; If the matching degree calculation result is greater than or equal to the preset value, it is determined that there is a first dependency path matching the second dependency path in the paraphrase corpus, and it is determined that there is a paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus; If the calculated matching degree result is less than the preset value, it is determined that there is no first dependency path matching the second dependency path in the paraphrase corpus, and it is determined that there is no paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus; The traversing the first dependency paths in the paraphrase corpus and calculating the matching degree between the second dependency paths and each of the first dependency paths comprises: Obtain matching words of the first dependency path and the second dependency path, wherein the matching words are participles having the same dependency relationship and the same glyph; The number of matching words and the total number of words in the second dependency path are calculated, and the ratio of the number of matching words to the total number of words is used as the matching degree between the second dependency path and the first dependency path.
2. The method for automatically generating a paraphrase text according to claim 1, characterized in that: The collecting training corpus, generating a first dependency path according to the training corpus, and constructing a paraphrase corpus based on the first dependency path includes: Collecting training corpus, extracting key description information from the training corpus and identifying the correctness of the extracted key description information, and using the training corpus with correct key description information as target training corpus; Performing word segmentation processing on the target training corpus and performing part-of-speech tagging on the word segmentation processing results; Constructing a syntax tree based on the part-of-speech tagging results and converting the syntax tree into a dependency tree; The first dependency path is generated according to the dependency tree, and the paraphrase corpus is constructed based on the first dependency path.
3. The automatic generation method of paraphrase text according to claim 1, characterized in that: The obtaining of the sentence to be repeated, preprocessing the sentence to be repeated, and generating a syntax tree according to the preprocessed sentence to be repeated includes: Using a word segmentation algorithm to perform word segmentation processing on the sentence to be repeated to obtain a plurality of ordered word segments; Performing part-of-speech tagging on the word segments in sequence; Arrange the part-of-speech tagging results in order to generate a syntax tree corresponding to the sentence to be repeated.
4. The method for automatically generating a paraphrase text according to claim 3, characterized in that: The converting the syntax tree into a dependency tree and acquiring a second dependency path corresponding to the sentence to be repeated according to the dependency tree comprises: Starting from the leaf nodes of the syntactic tree, each of the word segmentation nodes is attributed to the node of the corresponding word class; Obtaining the main word of each part of speech, and attributing the main word in each part of speech to a parent node from bottom to top; Obtaining the central word of the sentence to be repeated, and attributing the central word to a root node to generate the dependency tree; Starting from the root node of the dependency tree, each of the word segments is encoded in sequence to generate a word sequence, and the second dependency path is generated according to the dependency relationship between each of the word segments in the dependency tree and the word sequence.
5. An automatic generation device for paraphrasing text, characterized in that: include: A collection module, used for collecting training corpus, generating a first dependency path according to the training corpus, and building a paraphrase corpus based on the first dependency path, wherein the first dependency path is used for describing the dependency relationship between words in a sentence; A generating module, used for acquiring a sentence to be repeated, preprocessing the sentence to be repeated, and generating a syntax tree according to the preprocessed sentence to be repeated; A conversion module, configured to convert the syntax tree into a dependency tree and obtain a second dependency path corresponding to the sentence to be repeated according to the dependency tree; A matching module, configured to match the second dependency path with the first dependency path in the paraphrase corpus, and using the training corpus corresponding to the matched first dependency path as the paraphrase text of the sentence to be paraphrased, comprises: Matching the second dependency path with the first dependency path in the paraphrase corpus, and judging whether there is a paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus according to the matching result; If there is a paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus, replacing the sentence to be paraphrased with a training corpus corresponding to the first dependency path; If there is no repeatable sentence of the sentence to be repeated in the repeat corpus, retaining the sentence to be repeated; The step of matching the second dependency path with the first dependency path in the paraphrase corpus and judging whether there is a paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus according to the matching result comprises: Traversing the first dependency paths in the paraphrase corpus, and calculating the matching degree between the second dependency path and each of the first dependency paths; Comparing the matching degree calculation result with a preset value, and determining whether there is a first dependency path matching the second dependency path in the paraphrase corpus according to the comparison result; If the matching degree calculation result is greater than or equal to the preset value, it is determined that there is a first dependency path matching the second dependency path in the paraphrase corpus, and it is determined that there is a paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus; If the calculated matching degree result is less than the preset value, it is determined that there is no first dependency path matching the second dependency path in the paraphrase corpus, and it is determined that there is no paraphraseable sentence of the sentence to be paraphrased in the paraphrase corpus; The traversing the first dependency paths in the paraphrase corpus and calculating the matching degree between the second dependency paths and each of the first dependency paths comprises: Obtain matching words of the first dependency path and the second dependency path, wherein the matching words are participles having the same dependency relationship and the same glyph; The number of matching words and the total number of words in the second dependency path are calculated, and the ratio of the number of matching words to the total number of words is used as the matching degree between the second dependency path and the first dependency path.
6. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for automatically generating a paraphrase text according to any one of claims 1 to 4 is implemented.
7. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for automatically generating a paraphrase text according to any one of claims 1 to 4 is implemented.
