A method and apparatus for identifying textual relationships

Abstract

This disclosure provides a method and apparatus for identifying textual relationships. The method includes: acquiring two text fragments; inputting the two text fragments into a textual relationship classification model for classification to obtain the textual relationship between the two text fragments; wherein the classification model is obtained by processing monolingual training text to obtain expanded data, obtaining final expanded data based on the expanded data, and training the model using the final expanded data. This disclosure achieves better model performance by automatically expanding textual data and training the model using the expanded data, thus resulting in more accurate textual relationship classification.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method and apparatus for identifying textual relationships. Background Technology

[0002] Discourse relationship identification methods refer to the task of identifying the discourse relationship between two given text segments. Specifically, given two text segments, such as (The weather was nice today, but it was terrible yesterday), the method predicts that the discourse relationship between them is a contrast. However, existing methods train discourse relationship classifiers based on labeled discourse corpora, but the scale of existing labeled corpora is very small. In addition, manually labeling discourse corpora is costly and difficult to scale up. Summary of the Invention

[0003] This disclosure provides a method and apparatus for identifying chapter relationships, which addresses the shortcomings of existing technologies such as small data scale and low efficiency, and achieves the effect of automatically expanding data and efficiently classifying data.

[0004] Firstly, this disclosure provides a method for identifying textual relationships, including:

[0005] Get two text fragments;

[0006] The two text fragments are input into the discourse relationship classification model for classification to obtain the discourse relationship between the two text fragments;

[0007] The text relationship classification model is obtained by processing monolingual training texts to obtain augmented data, obtaining final augmented data based on the augmented data, and training the model using the final augmented data.

[0008] According to the discourse relation identification method provided in this disclosure, the discourse relation classification model is obtained by processing monolingual training text to obtain expanded data, obtaining final expanded data based on the expanded data, and training using the final expanded data, including:

[0009] The monolingual training text is processed to obtain the restored text corresponding to the monolingual training text;

[0010] The restored text is filtered to obtain expanded data;

[0011] The augmented data is extracted at different ratios and mixed with the original standard data to obtain the corresponding mixed data;

[0012] The text relationship classification model is trained for the first time based on the mixed data to obtain the mixed data with the highest accuracy.

[0013] The augmented data extracted according to the proportion of the mixed data with the highest accuracy is determined as the final augmented data;

[0014] The text relationship classification model is trained a second time based on the final expanded data to obtain the trained text relationship classification model.

[0015] According to the discourse relationship identification method provided in this disclosure, the step of inputting two text fragments into a discourse relationship classification model for classification to obtain the discourse relationship between the two text fragments specifically includes:

[0016] The text segments are encoded using a bidirectional encoding model to obtain classification vectors for the text segments;

[0017] The classification vector of the text fragment is input into the classifier to obtain the chapter relationship between the two texts.

[0018] According to the text relation identification method provided in this disclosure, after classifying the text into a text relation classification model to obtain the text relation, the method further includes:

[0019] The chapter relationships are compared with the corresponding standard category labels to determine the cross-entropy;

[0020] The correctness of the chapter relationship is determined based on the magnitude of the cross-entropy.

[0021] According to the discourse relation recognition method provided in this disclosure, the step of processing the monolingual training text to obtain the restored text corresponding to the monolingual training text specifically includes:

[0022] Obtain the monolingual training text;

[0023] Extract adjacent first and second text segments from the monolingual training text;

[0024] Replace the words other than the class relation words in the first text segment with the first noise character to obtain the replaced first text segment;

[0025] Replace the words other than the class relation words in the second text segment with the second noise character to obtain the replaced second text segment;

[0026] The replaced first text fragment and the replaced second text fragment are concatenated to form the replaced text;

[0027] The replaced text is input into a pre-trained text restoration model to obtain the restored text corresponding to the monolingual training text.

[0028] According to the text relationship identification method provided in this disclosure, the step of filtering the restored text to obtain expanded data specifically includes:

[0029] The multiple restored texts are scored, and the score result of each restored text is obtained;

[0030] Based on the scoring results of each restored text, the M*K restored texts with the highest scores are selected;

[0031] K restored texts are randomly selected from the M*K restored texts as supplementary data.

[0032] According to the text relation identification method provided in this disclosure, the step of extracting the expanded data at different proportions and mixing it with the original standard data to obtain corresponding mixed data specifically includes:

[0033] For each restored text in the augmented data, the perplexity is calculated to determine the perplexity value of each restored text;

[0034] The expanded data are arranged in descending order according to the perplexity value of each restored text;

[0035] Select the top N percent of the restored text from the sorted augmented data as the data to be mixed.

[0036] The data to be mixed is mixed with the original standard data to obtain mixed data.

[0037] According to the text relation identification method provided in this disclosure, the text relation classification model is trained for the first time based on the mixed data to obtain the mixed data with the highest accuracy, specifically including:

[0038] The mixed data is divided into training set data and validation set data;

[0039] The passage relationship classification model is classified based on the training set data to obtain the trained passage relationship classification model.

[0040] The accuracy of the trained text relationship classification model is tested based on the validation set data to obtain the accuracy of the text relationship classification model.

[0041] The mixed data corresponding to the maximum accuracy is determined based on the accuracy rate.

[0042] According to the chapter relationship identification method provided in this disclosure, the step of determining the corresponding proportion of the augmented data extracted based on the mixed data with the highest accuracy as the final augmented data specifically includes:

[0043] Determine the proportion of the mixed data corresponding to the maximum accuracy.

[0044] The augmented data extracted according to the corresponding ratio will be used as the final augmented data.

[0045] According to the text relation recognition method provided in this disclosure, the method for obtaining the pre-trained text reconstruction model includes:

[0046] Obtain pre-trained monolingual text;

[0047] Extract adjacent third and fourth text segments from the pre-trained monolingual text, and concatenate the third and fourth text segments to form the pre-trained text before replacement;

[0048] Replace the words other than the class relation words in the third text segment with the first noise character to obtain the replaced third text segment;

[0049] Replace the words other than the class relation words in the fourth text segment with the second noise character to obtain the replaced fourth text segment;

[0050] The replaced third text fragment and the replaced fourth text fragment are concatenated to form a pre-trained replaced text;

[0051] The pre-trained replaced text is input into the text restoration model to obtain the restored text;

[0052] The restored text is compared with the pre-trained text before replacement;

[0053] If the restored text is different from the pre-trained text before replacement, the model parameters of the text restoration model are adjusted until the text restored by the text restoration model with adjusted parameters is the same as the pre-trained text before replacement.

[0054] The text restoration model that matches the pre-trained text before replacement is taken as the trained text restoration model.

[0055] Secondly, this disclosure provides a text relationship identification device, comprising:

[0056] The text fragment acquisition module is used to acquire two text fragments;

[0057] The classification module is used to input the two text fragments into the discourse relationship classification model for classification, and to obtain the discourse relationship between the two text fragments;

[0058] The text relationship classification model is obtained by processing monolingual training texts to obtain augmented data, obtaining final augmented data based on the augmented data, and training the model using the final augmented data.

[0059] According to the document relationship identification device provided in this disclosure, the classification module is further configured to:

[0060] The monolingual training text is processed to obtain the restored text corresponding to the monolingual training text;

[0061] The restored text is filtered to obtain expanded data;

[0062] The augmented data is extracted at different ratios and mixed with the original standard data to obtain the corresponding mixed data;

[0063] The text relationship classification model is trained for the first time based on the mixed data to obtain the mixed data with the highest accuracy.

[0064] The augmented data extracted according to the proportion of the mixed data with the highest accuracy is determined as the final augmented data;

[0065] The text relationship classification model is trained a second time based on the final expanded data to obtain the trained text relationship classification model.

[0066] According to the document relationship identification device provided in this disclosure, the classification module is further configured to:

[0067] The text segments are encoded using a bidirectional encoding model to obtain classification vectors for the text segments;

[0068] The classification vector of the text fragment is input into the classifier to obtain the chapter relationship between the two texts.

[0069] According to the document relationship identification device provided in this disclosure, a third processing module is further included, wherein the third processing module is further configured to:

[0070] The chapter relationships are compared with the corresponding standard category labels to determine the cross-entropy;

[0071] The correctness of the chapter relationship is determined based on the magnitude of the cross-entropy.

[0072] According to the document relationship identification device provided in this disclosure, the classification module is further configured to:

[0073] Obtain the monolingual training text;

[0074] Extract adjacent first and second text segments from the monolingual training text;

[0075] Replace the words other than the class relation words in the first text segment with the first noise character to obtain the replaced first text segment;

[0076] Replace the words other than the class relation words in the second text segment with the second noise character to obtain the replaced second text segment;

[0077] The replaced first text fragment and the replaced second text fragment are concatenated to form the replaced text;

[0078] The replaced text is input into a pre-trained text restoration model to obtain the restored text corresponding to the monolingual training text.

[0079] According to the document relationship identification device provided in this disclosure, the classification module is further configured to:

[0080] The multiple restored texts are scored, and the score result of each restored text is obtained;

[0081] Based on the scoring results of each restored text, the M*K restored texts with the highest scores are selected;

[0082] K restored texts are randomly selected from the M*K restored texts as supplementary data.

[0083] According to the document relationship identification device provided in this disclosure, the classification module is further configured to:

[0084] For each restored text in the augmented data, the perplexity is calculated to determine the perplexity value of each restored text;

[0085] The expanded data are arranged in descending order according to the perplexity value of each restored text;

[0086] Select the top N percent of the restored text from the sorted augmented data as the data to be mixed.

[0087] The data to be mixed is mixed with the original standard data to obtain mixed data.

[0088] According to the document relationship identification device provided in this disclosure, the classification module is further configured to:

[0089] The mixed data is divided into training set data and validation set data;

[0090] The passage relationship classification model is classified based on the training set data to obtain the trained passage relationship classification model.

[0091] The accuracy of the trained text relationship classification model is tested based on the validation set data to obtain the accuracy of the text relationship classification model.

[0092] The mixed data corresponding to the maximum accuracy is determined based on the accuracy rate.

[0093] According to the document relationship identification device provided in this disclosure, the classification module is further configured to:

[0094] Determine the proportion of the mixed data corresponding to the maximum accuracy.

[0095] The augmented data extracted according to the corresponding ratio will be used as the final augmented data.

[0096] According to the document relationship identification device provided in this disclosure, the classification module is further configured to:

[0097] Obtain pre-trained monolingual text;

[0098] Extract adjacent third and fourth text segments from the pre-trained monolingual text, and concatenate the third and fourth text segments to form the pre-trained text before replacement;

[0099] Replace the words other than the class relation words in the third text segment with the first noise character to obtain the replaced third text segment;

[0100] Replace the words other than the class relation words in the fourth text segment with the second noise character to obtain the replaced fourth text segment;

[0101] The replaced third text fragment and the replaced fourth text fragment are concatenated to form a pre-trained replaced text;

[0102] The pre-trained replaced text is input into the text restoration model to obtain the restored text;

[0103] The restored text is compared with the pre-trained text before replacement;

[0104] If the restored text is different from the pre-trained text before replacement, the model parameters of the text restoration model are adjusted until the text restored by the text restoration model with adjusted parameters is the same as the pre-trained text before replacement.

[0105] The text restoration model that matches the pre-trained text before replacement is taken as the trained text restoration model.

[0106] Thirdly, this disclosure also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the above-described text relation identification methods.

[0107] Fourthly, this disclosure also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the document relation identification method as described above.

[0108] The text relationship recognition method and apparatus provided in this disclosure train a text relationship classification model using the following method: processing monolingual training text to obtain the corresponding restored text; then filtering the restored text to obtain expanded data; selecting the expanded data with the best training effect for the model from the expanded data, and training the model accordingly. This achieves automatic expansion of training data while simultaneously using the expanded data to better train the model, resulting in a more accurate classification effect. Furthermore, two text fragments are obtained; these two text fragments are input into the text relationship classification model for classification to obtain the text relationship between the two text fragments. This disclosure achieves accurate classification of text relationships through the above steps. Attached Figure Description

[0109] To more clearly illustrate the technical solutions in this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0110] Figure 1 This is a flowchart illustrating the text relationship identification method disclosed herein;

[0111] Figure 2 This is a schematic diagram of the text retrieval process for identifying chapter relationships provided in this publication;

[0112] Figure 3 This is a schematic diagram illustrating the acquisition of the text reconstruction model for chapter relationship recognition provided in this publication;

[0113] Figure 4 This is a schematic diagram of the structure of the document relationship recognition device disclosed herein.

[0114] Figure 5 This is a schematic diagram of the structure of the electronic device provided in this disclosure. Detailed Implementation

[0115] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the embodiments of this disclosure.

[0116] The following is combined with Figure 1 Description of Embodiments of this Disclosure This disclosure provides a method for identifying textual relationships, including:

[0117] Step 100: Obtain two text fragments;

[0118] Specifically, discourse relation identification refers to the task of identifying the discourse relationship between two given text segments. For example, the first text segment states: "The weather is very nice today"; the second text segment states: "Yesterday was very bad." The predicted discourse relationship between these two text segments is: transition. Therefore, before identifying the discourse relationship, it is necessary to obtain two text segments.

[0119] Step 200: Input the two text fragments into the discourse relationship classification model for classification to obtain the discourse relationship between the two text fragments;

[0120] The text relationship classification model is obtained by processing monolingual training texts to obtain augmented data, obtaining final augmented data based on the augmented data, and training the model using the final augmented data.

[0121] Specifically, in this invention, expanded data is obtained using training text. The expanded data with the best training effect is then selected as the final expanded data. This final expanded data is then used to train a text relationship classification model. Finally, the trained text relationship classification model is used to determine text relationships.

[0122] The text relationship recognition method provided in this disclosure trains a text relationship classification model using the following method: processing monolingual training texts to obtain corresponding restored texts; then filtering the restored texts to obtain expanded data; selecting the expanded data with the best training effect from the expanded data and training the model accordingly. This achieves automatic expansion of training data while simultaneously using the expanded data to better train the model, resulting in a more accurate classification effect. Furthermore, two text fragments are obtained; these two text fragments are input into the text relationship classification model for classification to obtain the text relationship between the two text fragments. This disclosure achieves accurate classification of text relationships through the above steps.

[0123] According to an embodiment of this disclosure, a method for identifying textual relationships includes processing monolingual training text to obtain expanded data, obtaining final expanded data based on the expanded data, and training using the final expanded data. Specifically, this includes:

[0124] The monolingual training text is processed to obtain the restored text corresponding to the monolingual training text;

[0125] The restored text is filtered to obtain expanded data;

[0126] The augmented data is extracted at different ratios and mixed with the original standard data to obtain the corresponding mixed data;

[0127] The text relationship classification model is trained for the first time based on the mixed data to obtain the mixed data with the highest accuracy.

[0128] The augmented data extracted according to the proportion of the mixed data with the highest accuracy is determined as the final augmented data;

[0129] The text relationship classification model is trained a second time based on the final expanded data to obtain the trained text relationship classification model. Specifically, the two obtained text fragments are concatenated into a single text, which is then input into the text relationship classification model for classification to obtain the text relationship.

[0130] The training of the text relationship classification model is achieved through the following steps:

[0131] To overcome the deficiency of insufficient data for model training in existing technologies, the text of the monolingual training text is restored after replacement before training the model, thereby increasing the training data for the model.

[0132] Then, the obtained restored text is mixed with the original standard data (such as the PDTB dataset) in different proportions. Since the restored data of different proportions have different training effects on the text relation classification model, the restored data of the proportion with the best training effect is selected.

[0133] Then, the restored data with the best training effect is selected to train the text relationship classification model separately, so as to obtain a trained text relationship classification model, and this text relationship classification model is used to judge the text relationship of the input text.

[0134] According to an embodiment of this disclosure, a method for identifying textual relationships includes, in which two text fragments are input into a textual relationship classification model for classification to obtain the textual relationship between the two text fragments, specifically including:

[0135] The text segments are encoded using a bidirectional encoding model to obtain classification vectors for the text segments;

[0136] The classification vector of the text fragment is input into the classifier to obtain the chapter relationship between the two texts.

[0137] Specifically, BERT (Bidirectional Encoder Representations from Transformers) is a bidirectional encoding model that serves as an alternative to Word2Vec. It has significantly improved accuracy in 11 directions within the NLP field. Essentially, BERT learns a good feature representation for words by running a self-supervised learning method on massive amounts of corpus data. BERT is built upon the transformer architecture and possesses powerful language representation and feature extraction capabilities. BERT's language input representation comprises three components.

[0138] Word embeddings divide words into a finite set of common sub-words, striking a balance between word effectiveness and character flexibility. Segmentation embeddings encode word positional information into feature vectors; position embeddings are crucial for introducing word positional relationships into the model. Position encoding tensors are used to distinguish between two sentences, such as whether B follows A (in dialogue or question-and-answer scenarios). For a sentence pair, the first sentence has a feature value of 0, and the second sentence has a feature value of 1. The final embedding vector is the sum of these three vectors.

[0139] Then, the classification vectors of the obtained text segments are directly input into the trained classifier, and the trained classifier is used to obtain the chapter relationship between the two text segments.

[0140] According to an embodiment of this disclosure, a method for identifying textual relationships includes, after classifying the text in a textual relationship classification model to obtain the textual relationships, the method further includes:

[0141] The chapter relationships are compared with the corresponding standard category labels to determine the cross-entropy;

[0142] The correctness of the chapter relationship is determined based on the magnitude of the cross-entropy.

[0143] Specifically, for a given event, there are n possibilities, each with a probability p(xi). The information content of a particular possibility can then be calculated. With the definition of information content, entropy is used to represent the expected value of all information. Relative entropy, also known as KL divergence, is used to measure the difference between two separate probability distributions P(x) and Q(x) for the same random variable x. P(x) is often used to represent the true distribution of the sample, i.e., the standard class label as referred to in this disclosure, while Q(x) represents the distribution predicted by the model.

[0144] In machine learning network training, the input data and labels are often already determined, so the true probability distribution P(x) is also determined, and information entropy is a constant here. Since the KL divergence represents the difference between the true probability distribution P(x) and the predicted probability distribution Q(x), a smaller value indicates a better prediction result. Therefore, it is necessary to minimize the KL divergence. Cross-entropy equals KL divergence plus a constant (information entropy), which is also called entropy and is used to represent the expected amount of information. Expectation is the sum of the probabilities of each possible outcome in the experiment multiplied by its result. The amount of information is inversely proportional to the probability of the information occurring. The higher the probability, the less information; the lower the probability, the more information. Because the cross-entropy formula is easier to calculate than the KL divergence formula, the cross-entropy loss function is often used to calculate the loss in machine learning.

[0145] Specifically, the following loss function is used in this disclosure:

[0146]

[0147] in, For standard category labels.

[0148] Cross-entropy measures the difference between two different probability distributions of the same random variable; in machine learning, it represents the difference between the true probability distribution and the predicted probability distribution. The smaller the cross-entropy value, the better the model's prediction performance.

[0149] Therefore, in this disclosure, the classification result obtained is determined based on the value of cross-entropy.

[0150] Combination Figure 2 As shown, according to an embodiment of this disclosure, a method for identifying textual relationships includes processing monolingual training text to obtain the restored text corresponding to the monolingual training text, specifically including:

[0151] Obtain the monolingual training text;

[0152] Extract the adjacent first text segment and second text segment from the monolingual training text;

[0153] Replace the words other than the class relation words of the first text segment with first noise characters to obtain the replaced first text segment;

[0154] Replace the words other than the class relation words of the second text segment with second noise characters to obtain the replaced second text segment;

[0155] Connect the replaced first text segment and the replaced second text segment into a replaced text;

[0156] Input the replaced text into a pre-trained text restoration model to obtain the restored text corresponding to the monolingual training text.

[0157] Specifically, given a large-scale monolingual text, extract every two adjacent sentences (two small clauses segmented by any punctuation), which are s1 and s2 respectively. Keep only the first three characters and the last character of s1, and replace the remaining words with a [M] to obtain s′1. For example, for the sentence "The weather is very good today", after replacement, it can be obtained as "The wea[M]good". Similarly, keep only the first three characters and the last character of s2, and replace the remaining words with a [M] to obtain s′2. For example, for the sentence "Yesterday was very bad", after replacement, it can be obtained as "Yester[M]bad". Concatenate s′_{1} and s′_{2} to get s′, that is, "The wea[M]good, Yester[M]bad". Use the text restoration model to perform a restoration operation on the sentence "The wea[M]good, Yester[M]bad" to generate a restored text different from the original input s, where s is obtained by concatenating s1 and s2, that is, s is "The weather is very good today, yesterday was very bad". The text obtained by the above restoration operation is different from s. The restored text can be sentences such as "The air is very good today, yesterday was extremely bad", "The sky color is very good today, yesterday was very bad", etc.

[0158] According to a discourse relation recognition method provided by an embodiment of the present disclosure, wherein, the screening of the restored text to obtain augmented data specifically includes:

[0159] Score the multiple restored texts to obtain the scoring results of each restored text;

[0160] According to the scoring results of each restored text, screen out the top M*K restored texts with the highest scores;

[0161] Randomly screen out K restored texts from the M*K restored texts as augmented data.

[0162] Specifically, to ensure the diversity of the generated text s, this invention uses beam search with secondary random sampling. During beam search, the top-10K candidates with the highest scores are first selected, and then the top-K candidates are randomly selected from the top-M*K candidates as the final output. The final K candidates for sentence terminators are the text augmentation results. Through this step, K times the amount of augmented data compared to the original data are obtained.

[0163] According to an embodiment of this disclosure, a method for identifying textual relationships includes extracting the augmented data at different proportions and mixing it with the original standard data to obtain corresponding mixed data. Specifically, this includes:

[0164] For each restored text in the augmented data, the perplexity is calculated to determine the perplexity value of each restored text;

[0165] The expanded data are arranged in descending order according to the perplexity value of each restored text;

[0166] Select the top N percent of the restored text from the sorted augmented data as the data to be mixed.

[0167] The data to be mixed is mixed with the original standard data to obtain mixed data.

[0168] Specifically, the perplexity of each restored text in the acquired augmented data is calculated, and then the augmented data is sorted in descending order of perplexity, taking the top 10%, top 20%, top 30%, ..., and then mixed with the original labeled data (such as the PDTB dataset).

[0169] According to an embodiment of this disclosure, a method for identifying textual relationships includes, in which a textual relationship classification model is trained for the first time based on the mixed data to obtain the mixed data with the highest accuracy, specifically including:

[0170] The mixed data is divided into training set data and validation set data;

[0171] The passage relationship classification model is classified based on the training set data to obtain the trained passage relationship classification model.

[0172] The accuracy of the trained text relationship classification model is tested based on the validation set data to obtain the accuracy of the text relationship classification model.

[0173] The mixed data corresponding to the maximum accuracy is determined based on the accuracy rate.

[0174] According to the text relation identification method provided in this disclosure, the step of determining the corresponding proportion of the expanded data extracted based on the mixed data with the highest accuracy as the final expanded data specifically includes:

[0175] Determine the proportion of the mixed data corresponding to the maximum accuracy.

[0176] The augmented data extracted according to the corresponding ratio will be used as the final augmented data.

[0177] Specifically, for example, after mixing the top 20% of the restored text with the original labeled data, the mixed data is divided into training data and validation data. After training the text relationship classification model based on the training data, the trained model is tested using the validation data to measure the accuracy of the text relationship classification model. If, after comparison, the text relationship classification model has the highest classification accuracy when using the top 20% of the restored text, then the top 20% of the restored text is used as the final augmented data to train the text relationship classification model, and the trained model is obtained. This trained model is then used to classify the input text.

[0178] Combination Figure 3 As shown, according to the text relationship recognition method provided in this disclosure, the method for obtaining the pre-trained text reconstruction model includes:

[0179] Obtain pre-trained monolingual text;

[0180] Extract adjacent third and fourth text segments from the pre-trained monolingual text, and concatenate the third and fourth text segments to form the pre-trained text before replacement;

[0181] Replace the words other than the class relation words in the third text segment with the first noise character to obtain the replaced third text segment;

[0182] Replace the words other than the class relation words in the fourth text segment with the second noise character to obtain the replaced fourth text segment;

[0183] The replaced third text fragment and the replaced fourth text fragment are concatenated to form a pre-trained replaced text;

[0184] The pre-trained replaced text is input into the text restoration model to obtain the restored text;

[0185] The restored text is compared with the pre-trained text before replacement;

[0186] If the restored text is different from the pre-trained text before replacement, adjust the model parameters of the text restoration model until the text obtained by restoring the pre-trained text after replacement according to the adjusted parameters of the text restoration model is the same as the pre-trained text before replacement;

[0187] Use the text restoration model corresponding to the case where the restored text is the same as the pre-trained text before replacement as the trained text restoration model.

[0188] Specifically, given a large-scale monolingual text, extract every two adjacent sentences (two small clauses segmented by any punctuation), which are s3 and s4 respectively. Keep only the first three characters and the last character of s3, and replace the remaining words with a [M] to get s′3. For example, for the sentence "The weather is very good today", after replacement, it can be obtained as "Today the [M] good", similarly, keep only the first three characters and the last character of s4, and replace the remaining words with a [M] to get s′4. For example, for the sentence "Yesterday was very bad", after replacement, it can be obtained as "Yesterday was [M] bad", splice s3 and s4 to get s; s is "The weather is very good today, yesterday was very bad". Splice s′3 and s′4 to get s′. That is, "Today the [M] good, yesterday was [M] bad", use s′ as the input and s as the output, and use the GPT model to train the text restoration model. Among them, when training the model, when s′ is used as the input and s can be output, the model training ends, if not, continue training.

[0189] Combined Figure 4 As shown, an embodiment of the present disclosure provides a discourse relation recognition device, including:

[0190] A text segment acquisition module 41, configured to acquire two text segments;

[0191] A classification module 42, configured to input the two text segments into a discourse relation classification model for classification to obtain the discourse relation between the two text segments;

[0192] Among them, the discourse relation classification model is obtained by processing monolingual training texts to obtain augmented data, obtaining final augmented data based on the augmented data, and training using the final augmented data.

[0193] Since the device provided by the embodiment of the present invention can be used to execute the method described in the above embodiment, and its working principle and beneficial effects are similar, details are not described here again. For specific content, refer to the introduction of the above embodiment.

[0194] The text relationship recognition device provided in this disclosure trains a text relationship classification model using the following method: It processes monolingual training text to obtain the corresponding restored text; then filters the restored text to obtain expanded data; from the expanded data, it selects the proportion of expanded data that best trains the model, and trains the model accordingly. This achieves automatic expansion of training data while simultaneously using the expanded data to better train the model, resulting in a more accurate classification effect. Furthermore, it acquires two text fragments; inputs these two text fragments into the text relationship classification model for classification, obtaining the text relationship between the two text fragments. Through the above steps, this disclosure achieves accurate classification of text relationships.

[0195] According to an embodiment of this disclosure, a document relationship recognition device is provided, wherein the classification module 42 is specifically used for:

[0196] The monolingual training text is processed to obtain the restored text corresponding to the monolingual training text;

[0197] The restored text is filtered to obtain expanded data;

[0198] The augmented data is extracted at different ratios and mixed with the original standard data to obtain the corresponding mixed data;

[0199] The text relationship classification model is trained for the first time based on the mixed data to obtain the mixed data with the highest accuracy.

[0200] The augmented data extracted according to the proportion of the mixed data with the highest accuracy is determined as the final augmented data;

[0201] The text relationship classification model is trained a second time based on the final expanded data to obtain the trained text relationship classification model.

[0202] According to an embodiment of this disclosure, a document relationship recognition device is provided, wherein the classification module 42 is specifically used for:

[0203] The text segments are encoded using a bidirectional encoding model to obtain classification vectors for the text segments;

[0204] The classification vector of the text fragment is input into the classifier to obtain the chapter relationship between the two texts.

[0205] A document relationship recognition device according to an embodiment of this disclosure further includes a third processing module, wherein the third processing module is used for:

[0206] The chapter relationships are compared with the corresponding standard category labels to determine the cross-entropy;

[0207] The correctness of the chapter relationship is determined based on the magnitude of the cross-entropy.

[0208] According to an embodiment of this disclosure, a discourse relationship recognition device is provided, wherein processing the monolingual training text to obtain the restored text corresponding to the monolingual training text specifically includes:

[0209] Obtain the monolingual training text;

[0210] Extract adjacent first and second text segments from the monolingual training text;

[0211] Replace the words other than the class relation words in the first text segment with the first noise character to obtain the replaced first text segment;

[0212] Replace the words other than the class relation words in the second text segment with the second noise character to obtain the replaced second text segment;

[0213] The replaced first text fragment and the replaced second text fragment are concatenated to form the replaced text;

[0214] The replaced text is input into a pre-trained text restoration model to obtain the restored text corresponding to the monolingual training text.

[0215] According to an embodiment of this disclosure, a text relationship recognition device is provided, wherein the step of filtering the restored text to obtain expanded data specifically includes:

[0216] The multiple restored texts are scored, and the score result of each restored text is obtained;

[0217] Based on the scoring results of each restored text, the M*K restored texts with the highest scores are selected;

[0218] K restored texts are randomly selected from the M*K restored texts as supplementary data.

[0219] According to an embodiment of this disclosure, a document relationship recognition device is provided, wherein the classification module 42 is specifically used for:

[0220] For each restored text in the augmented data, the perplexity is calculated to determine the perplexity value of each restored text;

[0221] The expanded data are arranged in descending order according to the perplexity value of each restored text;

[0222] Select the top N percent of the restored text from the sorted augmented data as the data to be mixed.

[0223] The data to be mixed is mixed with the original standard data to obtain mixed data.

[0224] According to an embodiment of this disclosure, a document relationship recognition device is provided, wherein the classification module 42 is specifically used for:

[0225] The mixed data is divided into training set data and validation set data;

[0226] The passage relationship classification model is classified based on the training set data to obtain the trained passage relationship classification model.

[0227] The accuracy of the trained text relationship classification model is tested based on the validation set data to obtain the accuracy of the text relationship classification model.

[0228] The mixed data corresponding to the maximum accuracy is determined based on the accuracy rate.

[0229] According to an embodiment of this disclosure, a document relationship recognition device is provided, wherein the classification module 42 is specifically used for:

[0230] Determine the proportion of the mixed data corresponding to the maximum accuracy.

[0231] The augmented data extracted according to the corresponding ratio will be used as the final augmented data.

[0232] According to an embodiment of this disclosure, a text relationship recognition device is provided, wherein the classification module 42 is specifically used for: acquiring pre-trained monolingual text;

[0233] Extract adjacent third and fourth text segments from the pre-trained monolingual text, and concatenate the third and fourth text segments to form the pre-trained text before replacement;

[0234] Replace the words other than the class relation words in the third text segment with the first noise character to obtain the replaced third text segment;

[0235] Replace the words other than the class relation words in the fourth text segment with the second noise character to obtain the replaced fourth text segment;

[0236] The replaced third text fragment and the replaced fourth text fragment are concatenated to form a pre-trained replaced text;

[0237] The pre-trained replaced text is input into the text restoration model to obtain the restored text;

[0238] The restored text is compared with the pre-trained text before replacement;

[0239] If the restored text is different from the pre-trained text before replacement, the model parameters of the text restoration model are adjusted until the text restored by the text restoration model with adjusted parameters is the same as the pre-trained text before replacement.

[0240] The text restoration model that matches the pre-trained text before replacement is taken as the trained text restoration model.

[0241] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include a processor 510, a communications interface 520, a memory 530, and a communication bus 540, wherein the processor 510, communications interface 520, and memory 530 communicate with each other via the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a text relationship recognition method. This method includes: acquiring two text fragments; inputting the two text fragments into a text relationship classification model for classification to obtain the text relationship between the two text fragments; wherein the text relationship classification model is obtained by processing monolingual training text to obtain expanded data, obtaining final expanded data based on the expanded data, and training using the final expanded data.

[0242] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of this disclosure, essentially, or the parts that contribute to the prior art, or parts of the technical solutions, can be embodied in the form of software products. These computer software products are stored in a storage medium and include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0243] On the other hand, this disclosure also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer, enable the computer to execute a text relationship recognition method provided by the methods described above. The method includes: acquiring two text fragments; inputting the two text fragments into a text relationship classification model for classification to obtain the text relationship between the two text fragments; wherein the text relationship classification model is obtained by processing monolingual training text to obtain augmented data, obtaining final augmented data based on the augmented data, and training using the final augmented data.

[0244] In another aspect, this disclosure also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a text relation recognition method provided above. The method includes: acquiring two text fragments; inputting the two text fragments into a text relation classification model for classification to obtain the text relation between the two text fragments; wherein the text relation classification model is obtained by processing monolingual training text to obtain augmented data, obtaining final augmented data based on the augmented data, and training using the final augmented data.

[0245] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0246] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0247] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit them. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure.

Claims

1. A method for identifying textual relationships, characterized in that, include: Get two text fragments; The two text fragments are input into the discourse relationship classification model for classification to obtain the discourse relationship between the two text fragments; The text relationship classification model is obtained by processing monolingual training texts to obtain expanded data, obtaining final expanded data based on the expanded data, and training using the final expanded data. The step of inputting the two text fragments into a discourse relationship classification model for classification to obtain the discourse relationship between the two text fragments specifically includes: The text segments are encoded using a bidirectional encoding model to obtain classification vectors for the text segments; The classification vector of the text fragment is input into a classifier to obtain the text relationship between the two texts; the text relationship classification model is obtained by processing monolingual training texts to obtain expanded data, and then obtaining final expanded data based on the expanded data, and training the model using the final expanded data, including: The monolingual training text is processed to obtain the restored text corresponding to the monolingual training text; wherein, the restored text is a new text different from the monolingual training text generated by restoring the replaced text; the replaced text is the text obtained by retaining the class relation words in the monolingual training text and replacing the remaining words with noise characters; The restored text is filtered to obtain expanded data; The augmented data is extracted at different ratios and mixed with the original standard data to obtain the corresponding mixed data; The text relationship classification model is trained for the first time based on the mixed data to obtain the mixed data with the highest accuracy. The augmented data extracted according to the proportion of the mixed data with the highest accuracy is determined as the final augmented data; The text relationship classification model is trained a second time based on the final expanded data to obtain the trained text relationship classification model.

2. The method for identifying textual relationships according to claim 1, characterized in that, After classifying the two text fragments into a discourse relationship classification model to obtain the discourse relationship between the two text fragments, the method further includes: The chapter relationships are compared with the corresponding standard category labels to determine the cross-entropy; The correctness of the chapter relationship is determined based on the magnitude of the cross-entropy.

3. The method for identifying textual relationships according to claim 1, characterized in that, The process of processing the monolingual training text to obtain the corresponding restored text specifically includes: Obtain the monolingual training text; Extract adjacent first and second text segments from the monolingual training text; Replace the words other than the class relation words in the first text segment with the first noise character to obtain the replaced first text segment; Replace the words other than the class relation words in the second text segment with the second noise character to obtain the replaced second text segment; The replaced first text fragment and the replaced second text fragment are concatenated to form the replaced text; The replaced text is input into a pre-trained text restoration model to obtain the restored text corresponding to the monolingual training text.

4. The method for identifying textual relationships according to claim 1, characterized in that, The process of filtering the restored text to obtain expanded data specifically includes: Score multiple restored texts and obtain the score result for each restored text; Based on the scoring results of each restored text, the M*K restored texts with the highest scores are selected; K restored texts are randomly selected from the M*K restored texts as supplementary data.

5. The method for identifying textual relationships according to claim 1, characterized in that, The process of extracting the augmented data at different proportions and mixing it with the original standard data to obtain corresponding mixed data specifically includes: For each restored text in the augmented data, the perplexity is calculated to determine the perplexity value of each restored text; The expanded data are arranged in descending order according to the perplexity value of each restored text; Select the top N percent of the restored text from the sorted augmented data as the data to be mixed. The data to be mixed is mixed with the original standard data to obtain mixed data.

6. The method for identifying textual relationships according to claim 1, characterized in that, The first training of the text relationship classification model based on the mixed data to obtain the mixed data with the highest accuracy specifically includes: The mixed data is divided into training set data and validation set data; The passage relationship classification model is classified based on the training set data to obtain the trained passage relationship classification model. The accuracy of the trained text relationship classification model is tested based on the validation set data to obtain the accuracy of the text relationship classification model. The mixed data corresponding to the maximum accuracy is determined based on the accuracy rate.

7. The method for identifying textual relationships according to claim 1, characterized in that, The step of determining the augmented data extracted according to the proportion of the mixed data with the highest accuracy as the final augmented data specifically includes: Determine the proportion of the mixed data corresponding to the maximum accuracy. The augmented data extracted according to the corresponding ratio will be used as the final augmented data.

8. The method for identifying textual relationships according to claim 3, characterized in that, The methods for obtaining the pre-trained text reconstruction model include: Obtain pre-trained monolingual text; Extract adjacent third and fourth text segments from the pre-trained monolingual text, and concatenate the third and fourth text segments to form the pre-trained text before replacement; Replace the words other than the class relation words in the third text segment with the first noise character to obtain the replaced third text segment; Replace the words other than the class relation words in the fourth text segment with the second noise character to obtain the replaced fourth text segment; The replaced third text fragment and the replaced fourth text fragment are concatenated to form a pre-trained replaced text; The pre-trained replaced text is input into the text restoration model to obtain the restored text; The restored text is compared with the pre-trained text before replacement; If the restored text is different from the pre-trained text before replacement, the model parameters of the text restoration model are adjusted until the text restored by the text restoration model with adjusted parameters is the same as the pre-trained text before replacement. The text restoration model that matches the pre-trained text before replacement is taken as the trained text restoration model.

9. A text relationship recognition device, characterized in that, include: The text fragment acquisition module is used to acquire two text fragments; The classification module is used to input the two text fragments into the discourse relationship classification model for classification, and to obtain the discourse relationship between the two text fragments; The text relationship classification model is obtained by processing monolingual training texts to obtain expanded data, obtaining final expanded data based on the expanded data, and training using the final expanded data. The classification module is also used for: The text segments are encoded using a bidirectional encoding model to obtain classification vectors for the text segments; The classification vector of the text fragment is input into a classifier to obtain the chapter relationship between the two texts; the classification module is further used for: The monolingual training text is processed to obtain the restored text corresponding to the monolingual training text; wherein, the restored text is a new text different from the monolingual training text generated by restoring the replaced text; the replaced text is the text obtained by retaining the class relation words in the monolingual training text and replacing the remaining words with noise characters; The restored text is filtered to obtain expanded data; The augmented data is extracted at different ratios and mixed with the original standard data to obtain the corresponding mixed data; The text relationship classification model is trained for the first time based on the mixed data to obtain the mixed data with the highest accuracy. The augmented data extracted according to the proportion of the mixed data with the highest accuracy is determined as the final augmented data; The text relationship classification model is trained a second time based on the final expanded data to obtain the trained text relationship classification model.

10. The document relationship recognition device according to claim 9, characterized in that, It also includes a third processing module, wherein the third processing module is further configured to: The chapter relationships are compared with the corresponding standard category labels to determine the cross-entropy; The correctness of the chapter relationship is determined based on the magnitude of the cross-entropy.

11. The document relationship recognition device according to claim 9, characterized in that, The classification module is also used for: Obtain the monolingual training text; Extract adjacent first and second text segments from the monolingual training text; Replace the words other than the class relation words in the first text segment with the first noise character to obtain the replaced first text segment; Replace the words other than the class relation words in the second text segment with the second noise character to obtain the replaced second text segment; The replaced first text fragment and the replaced second text fragment are concatenated to form the replaced text; The replaced text is input into a pre-trained text restoration model to obtain the restored text corresponding to the monolingual training text.

12. The document relationship recognition device according to claim 9, characterized in that, The classification module is also used for: Score multiple restored texts and obtain the score result for each restored text; Based on the scoring results of each restored text, the M*K restored texts with the highest scores are selected; K restored texts are randomly selected from the M*K restored texts as supplementary data.

13. The document relationship recognition device according to claim 9, characterized in that, The classification module is also used for: For each restored text in the augmented data, the perplexity is calculated to determine the perplexity value of each restored text; The expanded data are arranged in descending order according to the perplexity value of each restored text; Select the top N percent of the restored text from the sorted augmented data as the data to be mixed. The data to be mixed is mixed with the original standard data to obtain mixed data.

14. The document relationship recognition device according to claim 9, characterized in that, The classification module is also used for: The mixed data is divided into training set data and validation set data; The passage relationship classification model is classified based on the training set data to obtain the trained passage relationship classification model. The accuracy of the trained text relationship classification model is tested based on the validation set data to obtain the accuracy of the text relationship classification model. The mixed data corresponding to the maximum accuracy is determined based on the accuracy rate.

15. The document relationship recognition device according to claim 9, characterized in that, The classification module is also used for: Determine the proportion of the mixed data corresponding to the maximum accuracy. The augmented data extracted according to the corresponding ratio will be used as the final augmented data.

16. The document relationship recognition device according to claim 11, characterized in that, The classification module is also used for: Obtain pre-trained monolingual text; Extract adjacent third and fourth text segments from the pre-trained monolingual text, and concatenate the third and fourth text segments to form the pre-trained text before replacement; Replace the words other than the class relation words in the third text segment with the first noise character to obtain the replaced third text segment; Replace the words other than the class relation words in the fourth text segment with the second noise character to obtain the replaced fourth text segment; The replaced third text fragment and the replaced fourth text fragment are concatenated to form a pre-trained replaced text; The pre-trained replaced text is input into the text restoration model to obtain the restored text; The restored text is compared with the pre-trained text before replacement; If the restored text is different from the pre-trained text before replacement, the model parameters of the text restoration model are adjusted until the text restored by the text restoration model with adjusted parameters is the same as the pre-trained text before replacement. The text restoration model that matches the pre-trained text before replacement is taken as the trained text restoration model.

17. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the document relationship identification method as described in any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the document relation identification method as described in any one of claims 1 to 8.

Images